You ask ChatGPT a question that your business answers better than anyone. It gives a solid reply and lists three or four sources — none of them yours. You have a page on exactly that topic. It's more detailed than what it cited.
Most explanations jump straight to "do GEO" or "add schema markup." They skip the distinction that actually explains what's happening.
| Stage | What the AI is doing | What decides the outcome |
|---|---|---|
| 1. Retrieval | Whether to fetch your page at all, and whether the fetch succeeds | robots.txt, CDN rules, JS rendering, server response — a yes/no question |
| 2. Generation | Which of the retrieved sources get written into the answer | Whether your content contains a sentence worth quoting — a matter of degree |
Every "AI crawler check" tool out there — including ours — measures stage 1. It can tell you whether the door is open. It has no idea what happens at stage 2, because stage 2 isn't a configuration problem.
Your robots.txt says Allow: /. Real crawlers get a 403. This usually happens one layer up, at the CDN. Cloudflare has at least three settings that block AI crawlers silently while leaving your robots.txt looking perfect. We wrote up the whole thing separately, because it deserves its own page.
Plenty of people allow GPTBot and assume ChatGPT now has full access. But OpenAI split this into two crawlers:
GPTBot → collects content for model training OAI-SearchBot → live retrieval behind ChatGPT search
To be cited in search results, the one that matters is OAI-SearchBot. Allowing only GPTBot means you opted into training and out of citations. Perplexity has the same split: PerplexityBot for its index, Perplexity-User for live fetches when someone asks a question.
The names are nearly identical, which is exactly why this gets missed.
A client-rendered site hands crawlers an empty shell. Retrieval crawlers optimize for speed and generally do not execute JS — they won't wait for your framework to hydrate.
Check it yourself: right-click → View Page Source (not "Inspect") and search for a sentence from your article. If it isn't there, that's your problem. Or run curl -A "OAI-SearchBot" https://yoursite.com/ and read what comes back.
This is the most common way to fail stage 2, and the easiest to overlook.
Say someone asks whether compressing an image to 50 KB hurts quality. You wrote 2,000 words on it. But it opens with "In the realm of digital image processing, compression choices involve a complex interplay of factors…" — there is no sentence the model can lift into an answer. So it cites someone else.
Compare that to this:
That's a quotable block: a clear claim, a stated boundary, a concrete number, one sentence.
A quick self-test: how many sentences in your article still make sense when pulled out of context? If the answer is zero, length won't save you.
Trust assessment leans heavily on entity recognition — who runs this site, what they do, whether other sources corroborate. Plain text makes the model guess.
The cheapest fix is JSON-LD: Organization (who you are), Article with author (who wrote it), Product or FAQPage (what the page covers). This isn't a ranking trick — it's lowering the cost of understanding you.
There are 500 articles online saying the same thing. Yours is number 501. The model has no reason to pick it.
Information gain doesn't need to be dramatic, but it has to be real: a number you measured, a failure case nobody documented, a step-by-step only you can write, a conclusion that contradicts the consensus. Generic summaries are the least valuable form of content in AI retrieval, because summarizing is the one thing the model already does.
This one has to be said plainly: even with all six of the above handled, citation is not guaranteed. What gets cited depends on how the question was phrased, what was discussed earlier in the conversation, which candidates got retrieved, and which model version answered. Ask the same question twice and you can get different sources.
SEO works because rank is observable. Position 3, position 11 — measurable, comparable, reproducible. AI citations have no equivalent:
So there are exactly two things you can actually observe:
The workable strategy isn't "chase citations." It's make your quotable blocks solid, open the door, then wait. Treat it as a probability, not a KPI.
curl -A "OAI-SearchBot" -I https://yoursite.com/ and check for 200