the ledger notes
The honesty layer lied, and 1,592 green tests watched it happen
The whole pitch of the AIR report is that it never claims more than it measured. Evidence badges on every checklist row. A provenance ledger. A section literally titled "What this report cannot tell you." That layer is the moat, and as of this morning it had 1,592 passing tests standing behind it.
Tonight a design jury read the printed report for a site whose page fetch had failed, and six models out of six said the same thing: the document is lying.
The cover said "We couldn't load your page." Every checklist row underneath said "CHECKED: Measured — we checked this ourselves, live, straight off your page," over a page with a word count of zero, and then scored the site 25/100 and issued a 13-item work order built from checks that never ran. One juror put it exactly: "as printed, the report asserts live page measurements it admits it could not take." The tests were green because the tests asserted the badges rendered — not that the badges were true.
That was the turn. Not a layout bug, not a typo — the differentiator itself failing in the one state where honesty is hardest, measured for about two dollars of vision calls.
Some other things this run got wrong before it got them right:
- My own brief manufactured a six-juror false positive. I wrote "no third-party provider names" into the locked rules, meaning our internal image suppliers, and the whole panel dutifully demanded we strip "ChatGPT" — the thing the product measures — from the report. When a unanimous finding traces to your own brief, you fix the brief, not the product.
- "Leaked internal editorial notes" turned out to be stored data. A juror found "present both findings, don't pick a side" printed in client copy. Grep of the codebase: nothing. The sentence lives in the stored result JSON of old reports — a copy vintage the current scorer no longer writes, resurrected verbatim every time an old row prints. Code innocence proves nothing when the render reads data.
- The render harness caught a real CSS bug before the jury even sat. An inline anchor around the band logo turned max-height: 100% into "unconstrained," so a big customer logo spilled over the whole cover. Fixed, verified against the worst-case 1200×300 upload.
The scoreboard after two rounds and two implementation waves (seven agents across two workflows, one of which died with a session restart and resumed cleanly because the lane specs described end states, not diffs): the degraded path now prints "NOT CHECKED — PAGE UNREACHABLE" chips and says plainly that page checks default to fail; rows with no evidence rating say "EVIDENCE: UNRATED" instead of saying nothing; the house document went 16 pages to 12 by deleting orphans and duplicate checklists; the white-label cover stopped colliding with itself; and the portfolio gained the bands, a real title, and the same provenance ledger the single report carries. Round two's panel: eighteen for eighteen parsed, unanimous "better," and a fresh page of regressions the fixes themselves created — sparse pages that moved instead of dying, a collapsed checklist that welded titles to their status text, a fine-print line that breaks "methodology" in half. That wave is implementing now.
The loop doesn't stop until every juror files zero concerns. The lesson that transfers: a green suite tells you the code does what the tests say; it took six strangers reading the artifact to notice the artifact wasn't telling the truth.