The independent recount

A method that has worked once has not been proven. It might just have suited that particular dataset. So when two adjacent datasets came up — smaller, different landscape, five source authorities I hadn't worked with — I treated it as a test of the approach rather than a fresh build.

I was handed a starting list of 205 organisations and asked to verify it. The tempting reading of that task is “fill in the missing details”. The correct reading is “find out whether these 205 are the right 205”, and those are almost unrelated jobs.

A list you are given is a claim, not a fact — including when the organisation that published it is the one that governs the thing being listed.

Three commitments, made before any research started, and written into a specification document so I couldn't quietly relax them later when they got expensive.

• Confirm against the organisation's own site — never a governing list or a results bracket alone Carried over from the previous build, where an authority's own calendar listed an entity whose season had been cancelled, and several that genuinely participated filed no results at all. Both checks are needed, every time.
• Never infer a contact address, and never take one from a broker A guessed address that bounces is worse than a labelled blank, because it looks verified. If a search snippet offers an address that doesn't appear on the page itself, discard it — on the previous build that happened repeatedly and every one was wrong.
• Recount the total independently, at the end, without showing the second pass the first answer Not a review of my work. A separate derivation of the same number from the sources, so that agreement means something.

The specification also carried a section I'd recommend to anyone doing repeat work: “Traps that cost real time last build”. Writing down what went wrong the previous time is the cheapest engineering there is, and I had to relearn none of it.

1. Split by grouping and run the passes simultaneously Each grouping is independent, so nothing waits on anything else. This is what makes a verification standard this strict affordable at all — the expensive part is wall-clock time, and parallelism buys it back.
2. Confirm participation from the entity itself Its own site, its own listing of what it actually does. A governing body's page answers a different question than the one I was asking, which turned out to matter enormously.
3. Judge the source, not just the record When a source contradicted several confirmed findings in the same direction, I stopped treating that source as evidence rather than arguing record by record. Three of five rows wrong the same way isn't bad luck, it's an unreliable source.
4. Recount independently A second pass re-derived the totals from the authorities without seeing my result. Disagreements were worked to resolution rather than averaged or noted.
5. Disclose anything reasoned rather than read One entity blocks every automated visitor there is. Its address was deduced from two independent signals rather than confirmed by looking at the page — so that is written on the row. It is the only one in the file, and the reader knows which.
6. Write the summary for the person who has to act on it Plain language, for a non-technical reader: what changed, what's missing and why, and what needs a phone call. A finding nobody can act on isn't finished.

205 became 206. A net change of one, which is the least interesting number on this page. Underneath it: one entity added, three removed, and a source I stopped trusting.

The finding I'd lead with in any interview: a governing body's own current list was wrong, and not through sloppiness. Its page answered “who entered our event” while I was reading it as “who runs this programme”. Authoritative sources are authoritative about their own question, which is not always yours.

Two files with the same header have to mean the same thing

The recount caught something structural too. A column carrying the same name in both this dataset and the older one was pointing at two different kinds of page — in 175 of 206 rows it landed on the specific programme's page rather than the parent directory the column was defined to hold. All 206 were repointed, and a link was only accepted when the page both identified itself as a directory and listed several different activities. That test caught six more that looked fine at a glance.

The failure modes worth knowing

The trap list from the specification, generalised. Every one of these was paid for once already:

The last row is the same lesson as the previous build, in a new costume: a system that cannot tell “blocked” from “empty” will confidently report absences that don't exist.

I ran the recount once, at the end

It worked — but a recount at the end can only tell you that something is wrong, not where it entered. By then every grouping had been built the same way for weeks. If the recount had disagreed badly, I'd have had no way to narrow it down except redoing the lot.

A sampled recount running alongside the build — a handful of records per grouping, checked independently as each pass finishes — would have cost almost nothing and would have caught a systematic error while it was still one grouping wide.

I recorded source reliability in prose instead of in the data

I worked out that one reference source was wrong in a consistent direction, and I wrote that down — in a summary document, in a sentence, for a human to read and remember. Which means it's already half-lost.

I discovered something reusable about a source and stored it somewhere only a person can find.

That belongs on the row: which source confirmed this, and how much that source has historically been worth. Then the next build inherits the judgement instead of re-earning it.

“Reasoned rather than read” should be a field, not a sentence

Exactly one row in the file was deduced rather than confirmed by looking, and I disclosed it properly — in the summary. But a written note doesn't survive the file being copied, filtered or handed on, and that's the row where a future reader most needs to know.

It should have been a typed confirmation method column on every row: read directly, inferred from two signals, or unconfirmed. I already record where a value came from; I should equally record how sure I am, in a form that can be filtered.

And the thing I'd keep, without changing a line

Writing the “traps that cost real time last build” section before starting. It took maybe an hour, it read like admin, and it meant six expensive failure modes cost nothing the second time. Every repeat project should open with the list of ways the last one hurt.