Field notes from the control room · Draft

The Evidence Folder That Was Very Thorough

A composite story from a mid-size organisation.

Before the audit, a team built a folder called Evidence. It was a joy. Screenshots, exports and documents, named in one consistent style, dated, sorted into sub-folders that matched the controls. Someone had even added a contents page. It was the kind of folder that makes a person want to hug a spreadsheet.

The auditor opened it, scrolled for a while with visible respect, and asked a very small question: "Which system is this screenshot from?"

The turn

The answers came one by one. This one was from a test environment, because production was harder to reach. This one was taken the week before the audit, for a setting that was said to have been the same all year. This one showed a different control from the one in its file name. These two were pictures of a document that described a control, rather than the control doing anything.

Then the auditor asked for one sample pulled live from the system of record, right now, while everyone watched. The room discovered, together and quite warmly, that nobody knew how to produce it on demand.

The lesson

The folder proved diligence. It did not prove control. A screenshot is a claim about evidence: it says "this is what the system showed, once, to someone with a camera". Evidence is what the system itself records, and it can be produced again when asked, with its source and its time attached.

The team changed the shape of the work. The checks now run on a schedule and write their results straight from the system of record: what was checked, where it came from, when it ran, and what came back. A person samples a few of them against reality every so often, and a result that cannot be reproduced does not count. The folder shrank. The next audit conversation was shorter, and noticeably friendlier.

Evidence that cannot be reproduced from its source is a claim, not evidence.