Field notes from the control room · Draft

The Training Everybody Passed

A composite story from a mid-size organisation.

The annual awareness training was a short video and a short quiz. Completion was perfect, and the dashboard was a pleasing shade of green. There were congratulations. There may have been a cake.

The same month, the security team sent a harmless test message made to look like an overdue invoice, to see what would happen. A good share of the people who had just passed the quiz clicked it. It was sent at five to five on a Friday, which, in the team's defence, is also when real ones arrive.

The turn

Nobody was careless. The quiz asked what a suspicious link looks like, in a calm moment, with the answer sitting two lines above the question. The message asked something different: it asked a tired person, near the end of a long week, to deal with a small problem quickly. The training had measured whether people could answer. The test had measured what people do.

Someone plotted completion against clicks, and the graph looked like a cloud. A very polite, very green cloud.

The lesson

Completion measures attendance. Behaviour is what protects the organisation, and behaviour has to be measured as behaviour.

The team swapped the long quiz for short scenarios shaped like real work, sent often and different for each role, so the finance team sees invoices and the engineers see something about a pipeline. They put a clear report button where people already work, and they thanked whoever used it, publicly and by name of the role. The number on the dashboard changed too: it stopped counting who finished and started counting how quickly a suspicious message was reported. Clicking stopped being a confession and became a data point.

Count what people do, not what they finish.