the ledger notes
The council reviews a one-day-old CLI, and the brief itself gets caught lying
The air CLI was less than a day old — 180 lines, zero dependencies, shipped with one bug already found and fixed (the npm-link symlink guard that made the global binary exit silently, doing nothing, with code 0). Fresh code, fresh confidence. Good conditions for an adversarial review.
Seven seats convened: six frontier models over OpenRouter, chaired inside the harness with tools. The question was deliberately soft — "suggest the 5 highest-value things to implement or fix" — the kind of prompt that usually produces five plausible bullet points and no friction.
It produced friction in the first round.
The brief was wrong, and a seat caught it
The chair's brief stated, as a measured fact: "CLI polls every 20s, hard cap 180s, then returns whatever it has." Seraph (kimi-k3) read the actual source and filed it as a correction, not a suggestion: the 180-second cap lives only on the pending branch. The 429 rate-limit branch sleeps and continues forever — no elapsed-time check at all. A server in sustained overload could pin the CLI to a chair indefinitely.
The chair checked. Seraph was right. The brief's claim was retracted on the record — which matters, because every seat had been handed that false fact as VERIFIED and five of six had reasoned on top of it anyway.
The measurement that reshaped rank 2
The open research question was "how often is the 180s poll cap actually hit?" The chair ran it against the production database instead of letting anyone estimate: of 8,853 domain rows created in the last 48 hours, 117 resolved their ai-files check in under 60 seconds, zero resolved between 60 and 180 seconds, and 8,736 took longer than 180 — some nearly 40 hours.
The distribution isn't a distribution; it's a switch. When the hydrate queue is idle, first sight resolves inside one poll. When a backfill campaign is grinding, the job waits behind thousands of siblings and no client-side cap will outwait it. That killed two proposals at once (raising the cap, tuning the interval) and promoted a different one: when the cap expires, the CLI currently prints "AIR 0/10" and exits 0 — indistinguishable from a genuinely measured zero. Honesty at the cap became rank 2.
The owner kept moving the target, and that was fine
Mid-council, product direction kept arriving: default output stays colorful with icons and a --txt flag handles plain (which settled a five-way argument about Nerd-Font auto-detection — research had already found there is no standard for detecting one, just folklore heuristics). Then: keyword search is mandated — air 'best running shoes' should hit the search endpoint and disambiguate. Then: the CLI should be smart enough to tell a domain from a phrase without flags.
A council that can't absorb new constraints mid-flight is theatre. Round 3 took the directives as binding, re-ranked, and converged 4/5 (qwen starved on tokens in the final round — a known failure mode, recorded as absent, not as agreement).
The final five
- One wall-clock deadline over the whole lookup — pending polls and 429 sleeps both; nonzero exit on expiry. The unbounded loop was the only real landmine in the program.
- Pending-at-cap honesty — suppress the score headline while gathering, exit a documented 2, and make --json emit {data, meta} (it silently drops dataset_version today).
- Keyword/phrase search with smart detection — hostname-shaped args look up domains, everything else searches; many hits → show the top, ask to refine.
- --txt — and wire TERM=dumb/CI to it, honor NO_COLOR. The piped-bytes defect is real: glyph bytes reach grep today while colors correctly strip.
- A test harness for the untested I/O core — injected fetch/clock, and a spawn test of the symlinked entrypoint: the exact bug class that already shipped once.
What fell out: batch lookup, ETag persistence, score-as-exit-code (all demand- unmeasured), and the first-run wizard (deferred — --txt covers the need until a real user reports tofu).
Eight retractions across three rounds, including the chair's own. The cheapest evidence in the room was, as usual, a command someone actually ran: an od -c on piped output, a curl for an ETag header, one SQL query that turned "how often" into "never or always."
A parallel design jury restyled the same CLI while the council argued about its correctness — truecolor score gauge, braille spinner, clickable links. The two branches now have to merge, which is its own small lesson: review the function and the form separately and you buy yourself a merge, but both reviews come back sharper.