Why measurement stops at the moment of highest optimism

The most common point of evaluation is chosen for convenience, and it happens to be the point where the answer is most flattering.

Asking people what they thought of a session while they are still in the session is easy. Everybody is present, the response rate is high, the administrative cost is close to nothing, and the results arrive the same day. Asking the same people six weeks later what they have actually changed is hard. They have scattered, some have left, the ones who reply are the ones who liked it, and the answers are messy and difficult to summarise on a slide. Given a cheap flattering measure and an expensive ambiguous one, organisations reliably choose the first, and then treat it as though it answered the second question.

The result is a body of evidence that is systematically biased towards optimism. It is collected at the peak of goodwill, from a group still enjoying each other's company, about an experience they have not yet tried to use. High satisfaction at that moment is compatible with almost any outcome at work, including no outcome at all. It is not that satisfaction is worthless. A session people found confusing or patronising is unlikely to produce much. But it is a floor, not a ceiling, and it is routinely reported as though it were a verdict.

What follows from this is not a call for elaborate measurement machinery. Most organisations will never build that and probably should not try. What follows is a much smaller discipline: ask a different question at a different time. A short, specific enquiry a month later about what the person has done differently, directed at a handful of participants rather than all of them, will tell an honest buyer more than a thousand completed satisfaction forms. It will also, frequently, tell them something they would rather not hear, which is a large part of why it is not asked.