AIRANKS — The Authoritative Rankings for AI Web Content

AIRANKS measures AI visibility: we ask AI models real product and service questions, capture the complete answers as immutable observations, and publish what they contain — which brands were mentioned, which domains were cited, and which exact pages were linked. Every domain gets an AIR score from 1–10 (a decile of visibility in the active dataset; 0 means insufficient data), with the methodology in the open.

Skip to main content

AIRVER. FIGHTING — 2026

Artificial Intelligence Rankings

the ledger notes

The autoscaler was watching the wrong gauge

Build LogAugust 16, 2026 by Jeremy Schoemaker

Evening of cutover day. The one box left unchecked on the load-test list was the demo everyone assumes works because the console says so: push sustained load, watch CPU cross 60%, watch the auto-scaling group add a server. Textbook. We never got the textbook page. We got two better ones.

First attempt: the route got too fast to prove anything

The health route that did 160 requests/second at lunch did 530 at dinner — same fleet size, same route. I still can't fully attribute the 3.3× (candidates: a middleware-priority fix from the afternoon's review cycle, warmed opcache, or the lunch battery contending with its own preceding runs), and it's written down as measured-but-unexplained rather than claimed. The practical problem: at 530 rps the CPU peaked at 39%. You cannot demonstrate a 60% trigger with a route that cheap. Fine — aim the cannon at the expensive pages.

Second attempt: the fleet drowned at 19% CPU

The uncached domain pages are the heaviest thing we serve — full server render plus a stack of database queries, deliberately excluded from the microcache for CSRF reasons. Six continuous minutes at 300 concurrent connections, in-region. Result: 11.7 requests/second, median latency 14 seconds, 91% of requests failing — while web CPU sat at 19% and the database at 15%, with connections at 114 of a 317 ceiling.

Nothing was out of resources. Nothing scaled. Both facts are correct behavior, and that's the finding: the bottleneck was worker slots — 3 boxes × 4 Octane workers = 12 slots, each parked in I/O wait for most of its request's wall time. Throughput ceilings at roughly slots ÷ latency no matter how much silicon idles underneath, requests queue into timeouts, and a CPU-based autoscaler — correctly reading near-idle processors — adds nothing. The fleet would have drowned at 19% CPU without launching a single instance.

We set out to demonstrate the autoscaler works and instead demonstrated that on our heaviest route class it structurally cannot fire. The gauge was never wrong; it was the wrong gauge. The levers are now on file: workers ≫ vCPUs for I/O-bound loads, a request-count scaling policy alongside CPU, and a rethink of leaving the most expensive pages as the only uncached ones. Silver lining, measured: the rate-limit envelopes shed the abuse-shaped load, nothing crashed, and the site served 0.69-second pages within seconds of the cannon stopping.

The accidental headline: multi-AZ was fictional

Mid-burn, desired capacity ticked up — and the activity log finally coughed up a sentence that had apparently been repeating for hours: "t4g.small is not supported in us-east-1b." Every launch into our second availability zone had failed, silently, forever. Both "multi-AZ" instances lived in one AZ while the rebalancer looped: launch, fail, retry, fail. The morning's proud HA story was half true — redundant load balancer, single-zone fleet.

Fixed live, mid-burn: swap the dead zone's subnet for three that actually stock the instance type, and the first genuinely cross-AZ instance in the fleet's history launched while the test was still running. The burn's most valuable output wasn't a scaling chart. It was a sentence in an activity log nobody reads until desired-capacity does something weird.

And one near-miss for the permanent record

The night ended with a credential rotation, and the first version of the automation had a bug I almost didn't catch: [ success ] && deactivate-old-key || fallback-deactivate. Read it slowly — the fallback fires when the condition fails too, which means a failed fleet roll would have deactivated the key the un-rolled fleet was still signing mail with. The chain also happened to die on step one (an em-dash in an AMI description; AWS wants ASCII), which is what made me read the output instead of trusting the tail. Killed it with both keys intact, rebuilt it as an explicit if/else whose failure branch says it skipped the irreversible part.

Same shape as everything else today: the dangerous state never announces itself. The rebalancer failed quietly for hours. The fleet drowned with idle CPUs. The kill-switch fallback would have fired on failure. The only defense that worked, every time, was reading what the system actually said instead of what the automation implied.

← Back to blog