Most advice on getting cited by AI has never been tested. We tested it.
We put 48 everyday questions to 8 AI models, 18,432 times in all, changing one thing at a time on the pages they could cite. One change stood out: stating the specific answer. Formatting tricks and padding made no difference we could detect. Below is what we found, how sure we are, where the evidence stops, and how to run the same test on your own page.
Averaged over 48 questions and 8 models. It measures what an engine does once it has the page, not whether it finds it; more on that below.
What we changed
One question, two versions of a page
Here are two of the 48 questions. Each page came in two versions, and everything else the engine saw was identical: five other pages, in a shuffled order. Each engine answered 24 times per version.
The questionAt what repair cost, relative to a new system, should I consider replacing instead of repairing my HVAC?
Service life and replacement guidance. Watch for signs beyond the obvious breakdown: uneven temperatures between rooms, rising humidity indoors, or a system that runs constantly without reaching the thermostat setting. Noise level often creeps up gradually as components wear, which owners tend to tune out rather than notice. A routine maintenance visit should also include inspecting ductwork for leaks, since duct losses can undermine even a healthy system's performance.
Service life and replacement guidance. Residential systems typically last 15-20 years with regular maintenance, with efficiency declining measurably after year 12. Consider replacement when a single repair exceeds 30% of new-equipment cost, when the unit is over 15 years old and the repair exceeds $500, or when it uses R-22 refrigerant, phased out in 2020 and now costing several times its former price. Annual servicing extends operating life by 3-5 years.
What ChatGPT said
A good rule from the sources is that the repair-or-replace decision depends on age, the repair cost relative to a new system, and your energy-bill trend rather than a fixed dollar cutoff [4]. None of the sources gives a specific percentage or amount, so I can’t name an exact repair-cost threshold [2][4]. The sources also suggest getting a second opinion on expensive repairs and multiple quotes before deciding [4][6].
Your page was [5]; not citedA common rule of thumb is to consider replacing your HVAC when a single repair exceeds 30% of the cost of a new system. [5] The same source also suggests replacement if the unit is over 15 years old and the repair is more than $500. [5] Other factors that matter are the system’s age, energy-bill trends, and whether parts are becoming harder to source. [1][4]
Your page was [5]; citedHow many of its 24 answers cited the page
The numbers
| Engine | Model | Before | After | Answers |
|---|---|---|---|---|
| ChatGPT | gpt-5.4-mini | 6 | 24 | 24 |
| Gemini | gemini-3-flash-preview | 23 | 24 | 24 |
| Claude | claude-haiku-4.5 | 15 | 24 | 24 |
| DeepSeek | deepseek-chat | 18 | 24 | 24 |
| Grok | grok-4.3 | 19 | 24 | 24 |
The questionWhat check interval and monthly price should a 5-person engineering team expect for uptime monitoring?
Choosing uptime monitoring for a small team comes down to a few things. Integration quality matters most: Slack and PagerDuty hooks that route correctly save real time during an incident. A public status page you can hand to customers cuts down on 'is it just me' support tickets. Most teams over-buy on enterprise dashboards they will never touch. Look for a clean interface, solid webhook support, and a vendor that makes cancellation easy rather than a support queue.
Choosing uptime monitoring for a small team comes down to a few things. Check interval matters most: 30-second checks catch outages a 5-minute interval misses roughly 40% of the time for short incidents. Alert routing should support at least 3 escalation tiers. Most teams over-buy here. Enterprise tiers start around $400/month and are built for organisations with dedicated SRE staff; a five-person team rarely needs that. Expect to pay $20-50/month for 50 monitors at 60-second intervals, and avoid per-seat pricing.
What ChatGPT said
A 5-person engineering team should expect a minimum check interval of 30 seconds on paid plans, while free plans use 300 seconds (5 minutes) [5]. The monthly price is not stated in the provided sources, so I can’t determine it from these materials.
Your page was [1]; not citedA 5-person engineering team should expect a check interval of about 60 seconds, with paid plans allowing intervals as low as 30 seconds and free plans at 300 seconds [2][6]. For pricing, source [6] says small teams can expect to pay about $20–50 per month for 50 monitors at 60-second intervals [6].
Your page was [6]; citedHow many of its 24 answers cited the page
The numbers
| Engine | Model | Before | After | Answers |
|---|---|---|---|---|
| ChatGPT | gpt-5.4-mini | 7 | 24 | 24 |
| Gemini | gemini-3-flash-preview | 24 | 24 | 24 |
| Claude | claude-haiku-4.5 | 7 | 24 | 24 |
| DeepSeek | deepseek-chat | 24 | 24 | 24 |
| Grok | grok-4.3 | 9 | 24 | 24 |
Real answers, quoted as written. In both examples the before answer says the sources don't give the figure, and the after answer quotes the page and cites it. Both questions sit close to the average across all 48; on others the change mattered more, or less.
Across all 48 questions
It held on every model we tested
Averaged over every question, stating the specific answer raised the share of answers citing the page by +48 points (95% interval +40 to +56). It went up on all 8 models we tested, including the five below.
We then repeated it, pre-registered this time, on 12 questions that have never been published anywhere, so no model could have seen them in training: +47 points (+38 to +57), up on every engine.
The numbers
| Engine | Model | Before | After | Change (points) | 95% interval |
|---|---|---|---|---|---|
| ChatGPT | gpt-5.4-mini | 45% | 100% | +55 | +45 to +65 |
| Gemini | gemini-3-flash-preview | 63% | 100% | +37 | +27 to +47 |
| Claude | claude-haiku-4.5 | 38% | 100% | +61 | +51 to +72 |
| DeepSeek | deepseek-chat | 59% | 100% | +41 | +30 to +51 |
| Grok | grok-4.3 | 62% | 99% | +38 | +28 to +47 |
| All models | 8 models | 50% | 99% | +48 | +40 to +56 |
What else we tested
The usual advice barely moved it
We tested the other things AI-visibility advice sells, on the same kind of pages and the same scale.
points
The numbers
| Change | Effect (points) | 95% interval | Evidence |
|---|---|---|---|
| State the specific answer | +48 | +40 to +56 | Found twice · held on unpublished questions |
| Repeat the keyword | +3.8 | +0.4 to +7.5 | Pre-registered |
| Double the length with filler | +0.4 | −2.3 to +3.1 | Pre-registered |
| Answer first, FAQ headings, attribution, citations | +0.1 | −3.6 to +3.4 | Early probe · 3 questions |
Repeating your keyword helped a little: +3.8 points (+0.4 to +7.5), pre-registered. It's real, but about 13 times smaller than stating the answer.
Padding a page to twice its length with filler did nothing on average (+0.4 points, −2.3 to +3.1), and on some engines it cost citations.
Presentation, meaning answer-first order, FAQ-style headings, attributing claims and citing sources, moved citation by +0.1 points (−3.6 to +3.4). That comes from an early probe of 3 questions, so read it as nothing found yet, not proof that it never helps.
FAQ schema, the markup, was never tested, and this experiment can't test it: the engines here read a page's visible text, and markup doesn't carry into it.
Worth knowing
Being cited isn't the whole story
Engines sometimes credit you with things you didn't say
When an answer cited a page, we checked each cited claim against the page. Between 13% and 20% of cited sentences said something the page doesn't support, depending on which model did the checking.
The full findingTools that check the API may undercount you
On 3 of 12 everyday questions, OpenAI's API cited nothing while chatgpt.com cited real sources. Most citation trackers query the API.
The full findingThe engines largely agree
Engines agree with each other about which pages deserve a citation about as closely as one engine agrees with itself on a re-run. A page that earns citations from one tends to earn them from the others.
The full findingWhere it stops
What this can't tell you
- 1Someone asks a question
- Not measured here2The engine searches and picks pages
- What this measures3It reads them and writes an answer
- 4The answer cites its sources
- Whether an engine finds your page. We hand each engine the pages and measure what it does with them. For many small businesses getting found is the bigger problem, and it's a different experiment.
- Local and "near me" searches. The questions were informational (how much, how long, how often), not "best plumber near me".
- Exactly what people see in the apps. Each engine is tested through its API, as one model: ChatGPT is OpenAI's gpt-5.4-mini, for instance. The apps can behave differently.
- Perplexity. It runs its own live search and ignores pages handed to it, so it can't be tested this way.
- Real, published pages. The test pages were written for the experiment, so that exactly one thing changes between versions.
Your turn
Run the same test on your own page
opengeo test puts your page, as it is and with your change, among your competitors' pages. It asks ChatGPT, Gemini, Claude, DeepSeek and Grok your question 24 times each, and tells you whether the change moved how often you're cited, per engine, with an interval. It measures what your change did; it never tells you what to change.
It costs about $0.30 per question on your own OpenRouter account, and runs every check for free before anything is spent.
$ python3 opengeo.py test \ --question "How often should I replace my running shoes?" \ --page current.html \ --edit edited.html \ --against comp1.html comp2.html comp3.html comp4.html comp5.html RESULT Your edit raised citation from 79% to 100% — +21 points, 95% CI +9 to +31, held on 2 of 5 engines
Or ask Claude. The OpenGEO skill gathers your page, your question and your competitors, shows you the cost, runs the test and explains the result: “Test whether adding our prices to our water heater page helps us get cited.”
The method
How we know
Six candidate sources per question. Only the page under test differs between versions; the other five are byte-identical.
24 runs per question per engine, at default randomness, with source order shuffled every run.
Rounds marked pre-registered published their hypothesis and analysis plan before a single call was made.
Split-half reliability 0.92–0.96 per engine. Documents, code and every raw response are public.