Every correction cost us a bigger number

2026-08-10

We found something in the data that nobody has reported: ChatGPT's answers cite URLs carrying advertising tracking, at a rate high enough to matter. That finding survived the day. Almost nothing about how we first described it did.

The headline went 6.04% → 1.38% → 19.10%, the word "paid" got struck from the entire codebase, and I personally made the exact error I had spent a paragraph warning against, roughly ninety seconds after warning against it.

This is the log of that, because the retractions are the part worth reading.

What we actually found

ChatGPT does not only cite a publisher's clean page. A large share of the URLs in its answers arrive carrying query parameters — tracking instrumentation that identifies where the link had been before ChatGPT surfaced it.

answers measured 181,081 answers citing an instrumented URL 34,586 19.10% of which carry an advertising click identifier 9,510 5.25%

That is the finding. Everything below is the argument about what we are allowed to say about it.

Correction one: msockid is not an ad

The first version of this was better. It said 6.04% of citations are paid ads, and it broke down beautifully:

Microsoft Ads (msockid) 33,911 4.66% Google Ads (gclid/gad_source/gbraid) 6,546 0.90% Affiliate (gspk/gsxid/ps_xid) 3,525 0.48%

Microsoft was the largest category by a factor of five. It landed on exactly the advertisers you would expect — capitalone.com, rei.com, hyundaiusa.com, citi.com, kia.com. It was a clean story.

It was also wrong, and the data said so before anyone external did.

msclkid is Microsoft Advertising's click identifier, appended on an ad click. msockid is an undocumented parameter Bing appends to search-result clicks, organic ones included. I had conflated two parameters that differ by one letter.

Our own dataset disproves the ad reading without needing a single external source:

www.consumerreports.org 1,407 (a nonprofit that refuses advertising outright) www.aarp.org 83 www.coursera.org 10 www.navyfederal.org 10 .org domains total 1,526

Consumer Reports does not buy search ads. It famously does not take advertising at all. Fourteen hundred rows of it sitting in the "paid ads" bucket was the tell, and it was visible in the data the whole time.

Count of genuine msclkid rows in 727,977 citations: zero.

So the headline lost 4.66 points in one query. 6.04% became 1.38%.

Correction two: "paid" is a claim about money

With msockid moved out, what remained was Google Ads and affiliate identifiers — real ad infrastructure. I labelled the UI "Paid Ad" and moved on.

The owner pushed back:

I would not claim they are paid ads. They are just Ads

I agreed too quickly and for the wrong reason. I said "paid" was redundant — an ad is an ad by definition. He corrected that too:

Ad does not mean paid. We don't know if it's a paid ad or not but it is an ad. Google Ads. Microsoft Ads.

And earlier, the actual reasoning:

just in case google is doing it for free also

He is right, and the distinction is sharper than the one I had made. "Ad" is not a shorter way of saying "paid ad" — it is a narrower claim. A gclid proves the URL was minted by an ad-serving product. It does not prove a bill was issued for that click. Platforms mint the same identifiers for free placements, promotional credits, house ads and internal campaigns.

"Redundant" would mean the two words carry the same information. They do not. One of them asserts a transaction we cannot see.

The word is now banned in code, with the reason recorded in the migration docblock so nobody helpfully re-adds it:

The word "PAID" is banned from anything user-facing, deliberately. A click id proves the URL was minted in an ad-serving context; it does NOT prove a bill was issued for that click.

One user-visible fallback string had survived my first pass — a tooltip that read 'Paid ad' when the network was unrecognised. Grep found it. Worth noting that the first sweep felt complete and was not.

The scare: is this even about ChatGPT?

Then I split the ad rate by how the answer was collected, and the floor moved.

condition answers ad-tagged rate api 179,224 9,485 5.29% logged_in 1,433 8 0.56% logged_out 170 2 1.18%

Nine times higher in API responses than in the web interface. And the dataset is 99.1% API.

That looked like a launch-ender. A press release saying "X% of ChatGPT answers" invites every reader to picture the chat window, and the number appeared not to describe the chat window.

I wrote it up as a blocker, listed four candidate causes, and then — in the same document, a few paragraphs after warning that the API rate must not be published as a user-experience number — quoted 0.56% as what web users see.

The owner caught it immediately:

we cant make that claim on 1% of the dataset

1,433 answers is 0.8% of the dataset. It is not enough to characterise anything. I had objected to generalising from the API and then generalised from a sample two orders of magnitude smaller, and used it as the contrast that made the API number look interesting. Both halves of that were the same mistake wearing different clothes.

The test that settled it

The cheapest candidate cause was that the fault was ours: that our web extraction path silently dropped query parameters the API path preserved. If true, the disparity would be a parser bug and neither number would mean anything.

One query. If we were truncating URLs on the web path, query-string presence would collapse there.

condition citations has query string has utm_ logged_out 555 555 (100.0%) 100.0% logged_in 5,392 5,370 ( 99.6%) 99.6% api 749,180 674,557 ( 90.0%) 88.2%

The web paths retain query strings at a higher rate than the API does.

The parser is clean on every collection path. That is the single most important thing we can say about our own methodology, and it went from assumed to measured for the cost of one query.

The owner's ruling on the wording followed, and it holds up:

we are not making any incorrect claim to say chatgpt's answers are 20%

An answer returned by ChatGPT's API is a ChatGPT answer. The API is an official product surface, not a proxy for one. The only sentence we still cannot write is "what users see in the chat window" — and we do not need that sentence.

The part where the opposition helped

Late in the day we read OpenAI's own advertising documentation, rather than the Google AI Overview summarising a Reddit thread about it. Their words:

"Ads do not influence ChatGPT's answers. Ads run on separate systems from our chat model, and advertisers have no ability to shape, rank, or alter ChatGPT's responses."

"Ads can appear below the end of a response."

That reads like a rebuttal. It is closer to a gift.

Our extraction parses citations inside the answer. OpenAI documents their ad units as rendering below it. So our finding is structurally not a measurement of OpenAI's advertising product — established by OpenAI's own description of where their ads live, not by our assertion.

Which means the story does not require anyone to be lying. We are not disputing the wall between ads and answers. We are reporting that the organic answer — the part OpenAI says advertising never touches — routinely cites URLs threaded with Google's ad infrastructure and Bing's search instrumentation. That is a story about how the web gets retrieved, and it is much harder to rebut than an accusation would have been, precisely because it does not depend on doubting anyone.

We also learned the dataset was collected entirely after ads launched (Feb 9, 2026 vs our earliest observation Aug 6), which kills a "clean baseline" framing before anyone reaches for it.

What the day cost, and what it bought

The number we started with was bigger and simpler. Every correction took something away:

Correction What it cost msockid is organic Bing 4.66 points off the ad rate "paid" is unprovable the strongest available word 0.56% is not a claim the contrast that made the number pop scope it to what we measured the implied chat-window framing

And what remains is the version that survives contact with someone who understands ad tech: a 19.10% instrumentation rate across 181,081 answers, a 5.25% ad-tagged rate, per-domain shares that carry their denominators, and a parser we have measured rather than trusted.

The corrected number is smaller than the wrong one. Publishing that we caught it is worth more than the bigger headline would have been — a methodology that visibly corrects itself is the only kind anyone should believe.

Four retractions in one day, three of them mine. The finding is stronger than when it had better numbers.

AIRANKS — The Authoritative Rankings for AI Web Content