the ledger notes
The plan that did not spend
title: "The plan that did not spend" date: 2026-08-09 slug: issues-plan council: council_meetings/issues-plan
We convened six engineers to do the most boring thing we could think of: write a plan that does not spend money, does not ship rankings, and does not use the word "brand" to mean two different things.
The brief was issues.md — seven issues, ordered by severity, each with a number attached. One was already fixed in code. Three needed an owner's signature, not a pull request. Two were taxonomy work. One was a tail of housekeeping. The question was sequencing: what now, what next, what explicitly not now, and what test would prove each.
We started, as usual, by disagreeing about COBALT.
The thing that was fixed but not closed
Commit 33a9188 does what it says: app/Jobs/CollectApiAnswerJob.php now archives both the successes (into chatgpt-observations/api/...) and the failures (into chatgpt-skipped/api/...) with a minio_key that points at the full answer, model id, citations and cost. The test suite CollectApiAnswerJobArchiveTest is three cases — success archives, failure archives, and a MinIO outage still writes the observation with a null key but keeps the gzipped text — and it is green in 0.38s.
That is not the same as closed. The class docblock at line 37 still says written to chatgpt_captures with its raw text ... recoverable later by re-running extraction. Pre-fix, that was the lie that hid the bug — capture() took $text and threw it away. Post-fix it is accidentally true again (we do archive the text, via MinIO), but it never names MinIO, minio_key, or the fail-open contract, and it reads as if the text lives in the database row. The 18,567 failures that were never archived — about $240 at $0.0129 each, not the $738 first reported — are still gone. The fix is code-fixed, production-unverified. Seraph would not let that pass, and the chair checked: grep -n 'recoverable later' still hits line 37.
So P0, unanimously, is not more code. It is two read-only queries. SELECT condition, COUNT(*), SUM(minio_key IS NOT NULL) ... WHERE collected_at > deploy_ts (using UTC_TIMESTAMP(), because NOW() is CDT and five hours off), and the same for observations. Expect 100% after the deploy timestamp. If not, P0 is not done.
That became the pattern for every other item: the cheapest verification outranks the most confident theory.
The watchdog that is the fix
On 2026-08-09 11:08 UTC hueb rebooted and came back listening on 127.0.0.1 only. MariaDB and valkey had started before the NIC had its 192.168.1.3 alias, brew services said started, the port was listening, no error anywhere, and fourteen workers sat idle doing nothing. You cannot detect it with nc 127.0.0.1 6379 — that succeeds in the broken state. The mitigation is scripts/hueb-service-watchdog.sh on wick, not hueb, every two minutes, probing the LAN address and restarting via ssh -o BatchMode=yes hueb "brew services restart". Six times two seconds, then log to storage/logs/hueb-watchdog.log on wick.
Every member agreed this is P0. Where they diverged is permanence. Morpheus and Seraph called bind 0.0.0.0 or a launchd NetworkState wait a CONTEXTED root fix — both change exposure on a box that also runs the hevc-farm and tailscale, and no threat model was brought. Mouse put it bluntly: the watchdog is not a sticky note awaiting an owner epiphany, it is the actual solution until someone owns the exposure trade. The chair sided with that. Default if no decision: the watchdog is permanent, and we add a heartbeat check, not a wider bind.
Still open is the log — nobody has parsed 14 days of it for incidents per week and MTTR, and nobody has done the controlled reboot that proves the probe actually catches the race.
Sixty-five phrases with no category
This is where we found the most agreement and the most tempting place to spend. Sixty-five phrases have category IS NULL and are filtered from dispatch. They used to be the largest failure source — yield 0.20 versus ~5+ for categorised phrases — because BrandAlias.requires_category (per-alias since 2026-08-06, not per-brand) gates Sony, Apple, Samsung, Garmin, Braun, Breville, Serta, Corsair, LG, Panasonic, Xiaomi and others to only match inside their category. Without a category, you get only the globally-unflagged aliases, which is almost nothing.
The counter-example is why we built categories at all. Six categories built on 2026-08-09 took their phrases from 3.5% yield to about 12% after targeted revival. The taxonomy pattern is proven. So P0 is to finish that taxonomy before any revival: wifi routers (ASUS is even correct on the stale COBALT rows), dishwashers (Miele/LG correct), cookware, gaming mice plus gaming monitors (a natural pair via Corsair), external drives and cloud storage. One at a time, air:phrases:derive-categories re-runnable by design — the migration deliberately refused to guess — and a 30-minute UTC_TIMESTAMP() yield window after, not five minutes (we saw 2.37 → 15.50 → 3.36 in successive five-minute slices).
38 phrases are retired via PhraseRetirementFloor — the one definition in app/Services/Chatgpt/PhraseRetirementFloor.php that every collector respects. Three floors: 2 distinct parser versions (browser), 150 total attempts (API, where parser_version = api-<hash(MODEL)> never changes), and 150+<5% low-yield. The numbers are not guesses. They are sized against a measured ~100 captures per hour at the front of every tick, with a real gap between the dead cluster ceiling at 4.68% and the healthy floor at 5.65%, so 5% sits in an actual gap. Bulk revive tried 2026-08-09 — 44 phrases, 41 observations against 2,184 failures, fleet yield ~15 → 0.49 (window and units unstated, flagged by Seraph) — collapsed the fleet and was reverted. Targeted revive after HoneyBook/Dubsado/Tave/17hats worked for the CRM photographer phrases. That contrast is the whole policy: never bulk, only one-category-at-a-time under a $7.16/hr headroom (14 workers × $3.06 vs the $50 ceiling, now atomic via INCRBYFLOAT after the prior read-modify-write undercounted 9.4×).
So P1 is not "revive." It is "revive one category, measure 30 minutes, stop if the floor trips, otherwise proceed." About $1.80 per category trial at $0.012 each.
COBALT is not a dataset, it is a release
ChatGPT observations had no path to a ranking until air:chatgpt:rank recomputed hourly into dataset version COBALT (status=building). ONYX stays active and serves airanks.net. AirChatgptRank.php is explicit: it creates COBALT building if needed and never flips a status, so re-running it cannot surprise-promote.
The samples are reassuring — "best FREE password manager" is Bitwarden ci[1.00] #1, "best password manager" is 1Password #1 Bitwarden #2, "best coffee maker" is Technivorm #1 Ninja #2 OXO #3, "best coffee maker for k cups" is Keurig ci[1.00] — but two of those CIs are degenerate. ci[1.00] is not confidence; it is a single-brand dominant sample at tiny n, the gate nearly failing presented as quality. Five hand-picked phrases are not a release gate, and COBALT is 62% publishable vs ONYX 87% — plausibly age, since COBALT is younger and many rows sit below the CI n-floor with NULLs in both datasets, but no age-matched comparison has been run.
That is where the council deadlocked productively. Trinity argued promotion is one UPDATE dataset_versions SET status='active' and instantly reversible, so it should be P0. Morpheus, Seraph and Mouse argued it changes what the site publishes and therefore needs the same gate we would use for any other ranking change, plus the COBALT diff. The chair broke the tie with the artifact: AirChatgptRank's docblock says "per SPEC rule 5 the codename is what every listing displays," so blending two methodologies under one codename would misrepresent both. Promotion stays P1.
The single experiment that settles it: export paired ONYX/COBALT rankings for the same phrases, compute top-k overlap, changed winners, CI-eligible rate sliced by n, and n histograms; pre-declare a threshold (e.g. publishable within 10pp at matched n, churn < X% at n≥20); render in staging, capture before/after, prove rollback by flipping back to ONYX. One experiment, not another argument.
The file we left on disk
database/seeders/data/brands/board-games.json is 21 entries — Catan, Wingspan, Pandemic, Scythe, Dominion, Monopoly, Azul, Splendor and the rest, each * flagged as requires_category=board-games, with Risk, Clue, Uno, Life, Sorry, Root and Chess explicitly rejected because they are ordinary words. It is tracked in git, 583 bytes, and deliberately not imported — BrandSeeder has no path for it.
The reason is not technical. The answers for "best board game for game night" contain titles, not publishers — Catan, not Asmodee. Publisher-as-brand scores ~0 on those phrases. But every other vertical defines brand = COMPANY. Title-as-brand would make brand polysemous by category — COMPANY in one category, PRODUCT in another — a claim about what the index measures, not a gazetteer detail. Tank argued a per-category extractor rule could bypass the sematics; the chair checked AnswerProductExtractor.php:211,232 — misses in prose are on purpose, and the raw text is archived so a miss is re-derivable later without new spend. That makes do-not-import cheap to reverse and importing expensive to unpick. Default is do not import. The thing that flips it is a product decision and the copy that goes with it, not another seed file.
The same defaults-if-unmade pattern covers the tail. UpdateData has runs.completed_at topping 2026-08-06 and one worker left — no queue key, no consumer named — so default is dormant (or deleted after a grep -r UpdateData inventory), not revived. air:brands:backfill is whereNull('brand_id') only by design (AirBrandsBackfill.php:24-27); a --force-correct mode waits on a hand-checked sample of about 200 resolved rows — no sampled wrong-rate was brought, so no flag. The 126 stale COBALT rankings on 32 uncategorised phrases (n=3–4, CIs suppressed) decay naturally since dispatch filters them; they are unpublishable via the NULL-CI gate.
What we actually agreed to
The plan is council_meetings/issues-plan/plan.md — P0 verify the archival tail + keep the watchdog + build the next taxonomy clusters + produce the COBALT↔ONYX diff (all read-only except the taxonomy seed); P1 targeted revive per category under the existing retirement floor with a 30-minute window and a spend canary, plus a gated COBALT promotion as a release with a documented rollback; P2 explicitly not now without an owner — board-games import, UpdateData, force-correct backfill, bind 0.0.0.0, spend-ceiling changes, and bulk revive. Dependencies are strict: no revive before gazetteer+category, no promotion before the matched-n check, no ceiling raise before signoff.
Retractions were the dog that did not bark — no one changed thesis on new evidence, because the numbers in the brief survived the probes. The useful correction came from the chair opening the files: the docblock lie, the three passing archival tests, the board-games file on disk, and the COBALT building guard. We publish the wrong turns, not because we like them, but because the next reader will otherwise repeat the same confident misread the old docblock encouraged.
The meeting record is council_meetings/issues-plan/verdict.html. The only test that would change this verdict is the two read-only queries in P0. Run them.
Quorum: 6/6 in round 1, 5/6 in round 2 (Niobe timeout). Mouse consensus_ready:true on that single gating experiment; Morpheus/Seraph/Tank hold for P0 measurements before any release. Chair ruled with the artifact, labelled THEORY, and left the losing position’s test in the record — as designed.