AIRANKS — The Authoritative Rankings for AI Web Content

AIRANKS measures AI visibility: we ask AI models real product and service questions, capture the complete answers as immutable observations, and publish what they contain — which brands were mentioned, which domains were cited, and which exact pages were linked. Every domain gets an AIR score from 1–10 (a decile of visibility in the active dataset; 0 means insufficient data), with the methodology in the open.

AIR

BLOG

Skip to main content

the ledger notes

The panel tried to make up the numbers

Build LogAugust 20, 2026 by Jeremy Schoemaker

I gave six models the same commission: redesign the AIR header. Locked palette — green, white, black. Seven nav items, an AIR mark, a wordmark, a login pair. Told them what was wrong with the current one, measured: 248px tall at 1440, seven identically-weighted green caps in a centred row, no interaction vocabulary beyond an underline.

Then I anonymised all six designs to letters and had every model rank all six, including its own without knowing it.

Nobody guessed which one was theirs. Zero for six. Two ranked their own work first, one ranked its own dead last — and that one was unanimous last on all six ballots, for a reason four models named independently: its mobile breakpoint set the entire primary nav to display: none with no replacement. Convergent risk-naming is the most trustworthy thing a panel produces. When four strangers point at the same defect without conferring, it is a defect.

That round worked. I swapped two models and ran it again, and that is where it got interesting.

The graft nobody should have wanted

Round two's ballots carry a field called best_idea_to_graft — the one separable idea worth stealing from a losing design into the winner. Two of six models named the same thing: design A's identity topline, which read

CONFIDENCE INTERVAL ±0.42% · SAMPLE: 2.84M PROMPTS · DAILY INDEX: 98.4

One called it "the live confidence interval and prompt sample tape." Another said it "turns the product's differentiator into header furniture."

None of those numbers exist. The model invented them, marked them aria-hidden, and two of its peers wanted to copy them into the winning design.

A third juror killed it, and the wording is worth keeping:

It hardcodes fabricated telemetry into the masthead of a measurement product — aria-hidden decoration that a visitor reads as published fact, which is a credibility (and arguably legal) failure the moment it is wrong, and it is wrong the day it ships.

This is a product whose entire claim is that it publishes confidence intervals with every rank. A made-up confidence interval in the masthead is not a design flourish, it is the one thing we cannot ship. And the panel was one vote from recommending it — not out of malice, but because "put the differentiator in the header" is genuinely good design advice, and a model asked for good design advice will manufacture the data that makes the advice look implemented.

The lesson I am taking: when you ask a model to design a surface for a data product, it will fill empty space with plausible data. You have to say, in the brief, that inventing numbers is a disqualifying failure — and then you have to check anyway, because two of six will still want it.

Where my own numbers went wrong

I ran the two finalists as real implementations, in git worktrees, served locally. The complaint came back: too much white space. I measured the void and got 658px — 47% of the header width was empty.

I handed that to an agent to fix. It came back reporting the gap closed to 122–135px. I measured 439px. Neither of us was lying. It was measuring the raw span between two named endpoints; I was measuring the widest empty span between any two occupied boxes. It had dropped a faint dotted decoration into the void, which moved its number and not mine.

A decoration spanning a void is not the same as not having a void. The screenshot settled it in about two seconds — the band still read as empty.

The second attempt found the real cause, and it was better than the first: a flex-1 spacer and a fixed decoration in the same slot produce mathematically identical layouts, so that distance was invariant to the change being made. A measurement that cannot respond to what you are doing is not measuring what you are doing. justify-between across a track the nav already owned took it to 80px. The other design got to 54px the same way.

I also caught myself launching the "current" baseline server from the wrong worktree — a stray cd before a backgrounded command, so the control was serving one of the candidates while labelled current. Three ports, three asset hashes, all different: that is the check.

The retraction

When the founder picked the current header and told me to remove the rest, I deleted both branches and reported the work "genuinely gone — a rebuild, not a recovery."

That was wrong. I had grepped git reflog for the branch names, got nothing, and concluded the commits were destroyed. But git reflog with no argument shows HEAD's reflog, and deleting a branch never writes there. It was a probe that could only ever return nothing.

git cat-file -t aac59af → commit. All eight commits are still in the object store, recoverable for weeks. I told someone their work was unrecoverable on the strength of a question that could not have answered it.

Twelve headers, and we kept the one we had

Final tally: two rounds, six models, twelve designs, two of them built for real and measured across three pages.

header height widest void current 300px 32px (2%) Fighting Ledger Rail 147px 80px (6%) Stamp Index 115px 54px (4%)

Both candidates cut the header roughly in half. Both shipped a real Inertia loading rail. Both were accessible, tested, and green.

The founder looked at all three and kept the current one.

That is a legitimate outcome, and it is worth saying plainly, because the temptation after a run like this is to ship something to justify the effort. The panel measured well and argued well. It also produced a design that made up statistics about our own dataset, and two peers who wanted to spread them. Twelve designs is not twelve improvements, and a jury that agrees with itself is not evidence.

What actually shipped tonight was five commits that had been sitting undeployed since yesterday — the mobile nav among them — and ALB access logging, so the next time someone asks how many requests the toolbar is getting, the answer is a number instead of a shrug.

The first log batch landed four minutes later. Two of the forty-five rows were hits on a toolbar endpoint. Both were our own uptime monitor.

← Back to blog