Why AI Visibility Tools Show Different Results for the Same Brand
Two AI visibility tools can score the same brand far apart and both be right. What drives the gap, which differences are noise, and what to ask before you trust a number.
In this article
Your team runs two AI visibility trials in the same week. One dashboard says your brand shows up in a sliver of AI answers. The other says you appear in nearly half of them. Someone asks which tool is broken, and the honest answer is usually neither.
AI visibility tool results differ because each tool asks different questions, of different AI surfaces, a different number of times, and then counts the answers with its own formula. Once you know which of those four things moved, the gap stops being confusing and starts being useful. This post walks through each one, separates noise from signal, and ends with the questions to put to any vendor before a score reaches your board deck.
Why AI visibility tool results disagree
Every AI visibility score is built in the same broad way: pick a set of prompts, send them to one or more AI engines, record which brands and sources appear, and turn that record into a number. Each step involves a choice, and no two vendors make the same choices.
The prompt set decides what you see
The prompts are the sample frame. A tool that writes prompts from your category keywords will see a different market from one that uses synthetic prompts generated by a model, or one that draws on the questions your customers actually type.
Real people phrase the same need in very different ways. In SparkToro's January 2026 study, volunteers wrote 142 of their own prompts for one buying intent, and the average semantic similarity between any two prompts was 0.081. Two tools can therefore cover the same category and still be measuring different conversations.
Each tool reads a different surface
"AI search" is several products. ChatGPT, Gemini, Claude, Perplexity, Google AI Overviews, and Google AI Mode each assemble answers their own way, and even two Google surfaces disagree on sources. Ahrefs compared 540,000 query pairs from US data in September 2025 and found that AI Overviews and AI Mode reached similar conclusions (86% semantic similarity) while only 13.7% of their citations overlapped.
Collection method adds another layer. Some tools query a model through its API, others capture the consumer app a shopper sees, and location, language, and account history can all change an answer. Since June 2026, Google reports how often your pages appear in AI Overviews and AI Mode in Search Console's generative AI reports, but that view stops at Google and has no click data.
One run is a sample, not a reading
AI answers are generated fresh each time. SparkToro had 600 volunteers run 12 prompts through ChatGPT, Claude, and Google's AI a combined 2,961 times in November and December 2025. There was less than a 1 in 100 chance of getting the same list of brands twice, and less than 1 in 1,000 for the same list in the same order. SparkToro's findings also suggest running a prompt at least 60 to 100 times to see a model's real set of recommendations.
A tool that runs each prompt once a week and one that runs it many times a day are not measuring the same thing, even with identical prompts.
The score is a formula, not a fact
Tools also disagree on what counts. A mention means your name appears in the answer. A citation means one of your pages is linked as a source. A recommendation means the answer puts you forward as a choice. A brand can be cited without being named, or named without being recommended.
The denominator matters as much. A share of all tracked prompts reads lower than a share of only the answers that name any brand. The IAB's August 2026 guidelines note that more than 20 companies now sell AI visibility measurement, and they group the metrics into presence, prominence, portrayal, and persuasion so buyers can see which one a score describes.
Which differences are noise and which are signal
Not every gap deserves a meeting. Ordering inside a single answer is mostly noise, and so is a week-to-week move on a small prompt set. SparkToro started out expecting tracking to be pointless and changed its view: visibility measured across dozens to hundreds of prompts, each run many times, held up as a reasonable metric, while the rank position in any one answer did not.
The signal sits in three places:
- Your share of answers across a large, stable prompt set, run repeatedly
- The direction of that share over several weeks, not one reading
- The pattern by engine, since a brand can be strong in one engine and absent in another
When two tools disagree on the level but agree on the direction, believe the direction.
Questions to ask before you trust a score
Put these to any vendor, including the one you already use. If you are setting up tracking from scratch, our guide to monitoring your brand across ChatGPT, Perplexity, and AI search covers the setup.
- How many prompts are tracked, where did they come from, and do they stay fixed between reports?
- How many times is each prompt run, on which engines, and how often?
- Do you query the consumer apps or the APIs, from which locations, and logged in or out?
- What counts as an appearance: a mention, a citation, or a recommendation?
- What is the denominator behind each percentage?
- Can we read the raw answers behind every number?
A vendor who cannot answer the last question is asking you to trust the formula without the evidence.
How we measure it at Passionfruit
Our generative engine optimization work starts with hundreds of prompts built from a client's brand context and the questions their buyers ask, run across every major AI engine on a schedule. Every response is kept, with the brands, products, and sources it mentions, so a number on a report can always be traced back to the answers that produced it.
We report share of voice, average rank, sentiment, and citation share per engine against competitors, over time, and tie AI referrals to key events and revenue. For one cognitive wellness brand, presence in tracked AI answers rose from about 6% to 31% over six months on that kind of fixed prompt set, which is a trend you can act on.
Reading the gap the right way
Two AI visibility scores for the same brand are rarely a contradiction. They are two samples drawn with different methods, and the useful move is to pick one method you can inspect, hold it steady, and judge progress by direction across engines rather than by any single reading. For the research behind why answers shift, see our earlier analysis of why AI brand recommendations change with every query.
If you would like to see how your brand reads across the major engines on a prompt set you can audit line by line, book a call and we will walk through it with you.
Frequently asked questions
Why do AI visibility tools give different scores for the same brand?
They use different prompt sets, query different AI surfaces, run each prompt a different number of times, and count answers with different formulas. Any one of those choices can move the score, so two honest tools can land far apart.
Which AI visibility tool is the most accurate?
None is accurate in an absolute sense, because every score is a sample. The most useful tool is the one that shows its prompts, run counts, surfaces, and raw answers, and keeps that method steady over time.
How many times should you run a prompt to measure AI visibility?
Many more times than once. SparkToro's research suggests at least 60 to 100 runs of a prompt to see a model's real set of recommendations, and visibility measured across dozens to hundreds of prompts held up as a reasonable metric.
Is your rank inside an AI answer a meaningful metric?
Not from a single answer. SparkToro found less than a 1 in 100 chance of the same brand list twice and less than 1 in 1,000 for the same order, so position only means something as an average across many runs.
Can you compare AI visibility scores from two different tools?
Not directly. The prompts, surfaces, sampling, and denominators differ, so the levels are not comparable. When two tools disagree on the level but agree on the direction, believe the direction.
Do Google AI Overviews and AI Mode cite the same sources?
Mostly not. Ahrefs found only 13.7% of citations overlapped between the two across 540,000 query pairs, even though the answers reached similar conclusions, which is one reason tools that read different surfaces report different results.
