AI Search Simulation vs Real-World Performance: What Can You Actually Prove?
Simulated AI answers show what a model could say; real-world data shows what buyers saw and did. What each can prove, and how to run a test that holds up.
In this article
What does it mean when a vendor says your product is now "cited first" after a round of AI search simulation? It means a model, given prompts the vendor chose, under conditions the vendor controlled, answered that way when they tested it. That is useful information. It is not proof that buyers saw it, and it is not proof that it sold anything.
AI search simulation and real-world performance answer different questions, and most disagreements about GEO results come from mixing them up. This post sets out what each kind of evidence can prove, where each one stops, and how to design a test that your finance team will accept.
What AI search simulation actually measures
Simulation means sending prompts or scripted conversations to a model, through its API or its app, and scoring the answers. The prompts are usually synthetic, written by a person or generated by another model, and the test can run before anything is published, which makes it a fast way to compare versions of a page.
The limits are built in. Microsoft's Clarity team put it plainly in a March 2026 post on measuring AI visibility: simulated prompts show what a model could say, not what it did say to real users. A simulator also sees only the final answer. It misses the pages the engine retrieved and weighed but did not cite.
Simulation does have a strong place in research. The GEO paper from Princeton and IIT Delhi, accepted at KDD 2024, tested content changes on a 10,000-query benchmark using an engine the authors built from the top 5 Google results and a language model, and found that some methods lifted visibility by up to 40%. On Perplexity, tested with 200 samples, gains reached up to 37% on one of its metrics. That is the value of simulation: controlled comparisons. It is also the caveat: the gains were measured on test engines, not on your buyers.
Where simulation stops
Three things sit outside what any simulated test can show.
Real demand
No one outside the AI companies sees the prompts real buyers type. A synthetic prompt set is a model of demand, and it tends to be cleaner and shorter than the questions people actually ask. A product can win the simulated question and still be absent from the messy, specific ones.
Real conditions
Live answers depend on personalization, conversation history, location, device, and the model version on the day. A test run from a clean account in one location removes exactly the variation real buyers bring.
Cause and effect
A before-and-after simulation shows that a model reacted to your edit in that test. It does not show that the live engines changed for buyers, or that revenue followed. Proving that needs data from the real world and a comparison group.
What real-world data can prove
Evidence comes in layers, and each layer proves something the one before it cannot.
| Evidence | What it can show | What it cannot show |
|---|---|---|
| Simulated prompts and conversations | How a model answers chosen prompts under set conditions | Real demand or real exposure |
| Tracked prompts on live engines, run repeatedly | Your share of answers on a fixed set, and its trend | How often buyers really ask those prompts |
| Platform reports from Google and Bing | Real impressions or citations on those surfaces | Clicks, or engines they do not cover |
| Analytics and CRM | Visits, key events, and revenue from AI referrals | Influence that never produced a click |
| A controlled comparison | Whether a change likely caused the lift | Certainty for every engine and market |
How to run a test you can defend
A test that holds up looks more like a trial than a demo:
- Choose the pages to change, and a comparable group you will leave alone.
- Record a baseline for both: tracked prompts, Search Console and Bing data, and analytics, over the same weeks.
- Use simulation to pick the strongest version of each change before it ships.
- Ship the change to the test pages only, and keep everything else steady.
- After several weeks, compare test and control on tracked share, platform data, AI referrals, and revenue.
- Write down what the result proves and what it does not, before anyone builds a slide.
This is also why repeated tracking matters, which we covered in how to measure AI search visibility. A single before-and-after reading cannot carry a causal claim.
How we prove it at Passionfruit
In our generative engine optimization program, every action is annotated on the page it changed and measured after it ships, across tracked AI answers, platform data, and revenue. Simulation helps us choose what to ship; real-world data is what we report.
For one skincare device brand, that meant reporting two kinds of evidence side by side: a sixth-place rank among 304 tracked domains for AI answer visibility, at an 11.96% share of voice, and 17 purchases from AI answers in July 2026, recorded in analytics and led by ChatGPT. The first is a tracked estimate. The second happened. Conversations matter too: multi-turn AI search is where simulation earns its keep, since it can map follow-up paths that live data only shows after the fact.
Simulate to choose, measure to claim
AI search simulation is a good tool for deciding what to try. It is a poor tool for proving what worked. Use it to pick the changes, then let tracked answers, platform reports, analytics, and a control group tell you whether they paid off.
If you are weighing a simulation-led proposal and want to know what it would really prove, talk it through with us and we will map the evidence each claim needs.
Frequently asked questions
What is AI search simulation?
It is the practice of sending synthetic prompts or scripted conversations to AI models and scoring the answers, often before content is published. It shows how a model responds to chosen prompts under controlled conditions.
Can AI search simulation prove that GEO works?
Not on its own. Simulation shows that a model reacted to a change in a test, while proving results for real buyers needs tracked answers on live engines, platform data, analytics, and a comparison group.
What is the difference between synthetic prompts and real user prompts?
Synthetic prompts are written by people or generated by models to represent demand. Real user prompts are what buyers actually type, which outside tools cannot see and which tend to be longer and more specific.
What did the GEO research paper find?
On a 10,000-query benchmark and an engine built from the top 5 Google results, some content changes lifted visibility by up to 40%. On Perplexity, with 200 samples, gains reached up to 37% on one metric, measured on test engines rather than real buyers.
How do you prove an AI search change caused a result?
Change a set of pages and leave a comparable group alone, baseline both, and compare them after several weeks on tracked share, platform data, AI referrals, and revenue. That comparison carries the causal claim that a single before-and-after reading cannot.
Is AI search simulation still worth using?
Yes, as a good tool for deciding what to try. Use simulation to pick the strongest version of each change before it ships and to map follow-up paths, then let real-world data show whether the change paid off.
