You’ve bought a report. The numbers look good. Your business appears to be showing up in ChatGPT, your competitors are a couple of positions behind you, and there’s a quiet sense of reassurance.
Then you open ChatGPT yourself and type the kind of question your customers ask every day. And there they are, your competitors. Your business isn’t mentioned at all.
How can the report say one thing and reality say another? The answer comes down to how the data was collected. An AI visibility audit is only as reliable as the method behind it, and the differences between methods are larger than most people realise.
There are two ways to collect data from an AI tool
The first is via an API. You send a question directly to the underlying AI model and get a response back. It’s fast, scalable, and straightforward to automate.
The second is to scrape the interface your customers actually use. You open ChatGPT, Perplexity, Gemini or Copilot as any user would, ask the question, and record what actually appears on the screen.
The difference sounds minor. It isn’t.
Think of it this way: querying via API is like calling a business and speaking to someone in the warehouse. Scraping the interface is like walking into the shop and seeing what customers actually encounter. Same operation behind the scenes, completely different experience.
Why what you see in the interface is so different
ChatGPT, Perplexity, Gemini and Copilot are not just language models. They are products, and each product layers its own instructions, search integrations, geolocation and personalisation on top of the underlying model.
What we call “ChatGPT” carries a system prompt that users never see. It includes live web search, memory functions, IP-based location detection, and automatic query rewriting that happens before a response is even generated. None of that comes with you when you call the API.
Perplexity routes consumer queries through its own orchestration layer, which can select between several different AI models depending on the question. The API it offers exposes part of this. What’s built into the product is another thing.
Copilot is the clearest case of all. Microsoft has built out a set of Microsoft 365 Copilot APIs, with a Retrieval API generally available and Chat and Search APIs in public preview. But those are aimed at enterprise data inside Microsoft 365. There is no official API for the consumer Copilot your customers actually use.
Which of these do you think your customers are using?
What the data actually shows
Surfer SEO has compared the two methods in a study with data collected in August 2026. 1,000 prompts were sent to five AI services, both through the API and by scraping the interface real users see, for 13,779 answers in total.
The answers differed on almost every point. ChatGPT’s interface cited 12.1 sources per answer on average, the API 3.1. At the same time, the API wrote roughly twice as many words as users actually see.
But the most important figures were about overlap. Only 15.5% to 23.8% of brands appeared both in the API answers and in what users saw, and even after spelling variants of the same company were merged, the figure peaked at 31.6%. Of the domains ChatGPT cited, only 4.8% overlapped.
The study’s own conclusion is that the APIs and the apps disagreed on every single thing it examined.
For Perplexity, the overlap was higher but still low, 26.7% at domain level. And there the difference ran the other way: the API cited more sources than the app, 19.5 against 10.3.
What an API-based audit misses in practice
The technical gap has consequences that are very concrete for a small or medium-sized business.
If an audit identifies websites that ChatGPT “cites” about your industry based on API data, and you build your PR efforts around that list, you are working toward targets that barely exist in the real product. With under 5% source overlap, the odds are that you are pursuing the wrong domains entirely.
Nor do the differences all point the same way. ChatGPT’s API gives fewer sources than the app, Perplexity’s more. So API data can’t be converted into real answers with a simple factor, and your share of real AI responses may be stronger or weaker than the report suggests.
There is also a more technical issue worth knowing about. OpenAI removed GPT-4o from the ChatGPT interface on 13 February 2026, and by 3 April 2026 it was gone from every ChatGPT plan. It remains available in the API. An audit tool still running against GPT-4o via API is therefore testing a model that no ChatGPT user encounters any more.
Then there’s how ChatGPT actually searches. AirOps analysed 548,534 retrieved pages across 15,000 original prompts in a report published on 12 March 2026. ChatGPT generated its own follow-up searches on 89.6% of queries, and 95% of those follow-up queries had zero monthly search volume by traditional measures. Almost a third of the pages that were actually cited — 32.9% — appeared only in response to one of those follow-up queries, never to the original prompt. A single API call with a single prompt misses that entire surface.
The same report shows how severe the filtering is: 85% of the pages ChatGPT retrieves are never cited at all. Being found is not the same as being used.
A single run is never enough
Everything above is about the wrong data, collected the wrong way. But there is a separate problem that applies even to audits that scrape the right interface: randomness.
Rand Fishkin at SparkToro and a team at Gumshoe.ai published results on 28 January 2026 from 2,961 prompt runs across the same questions, carried out by 600 volunteer participants. Their finding: there is less than a one-in-a-hundred chance that two identical runs of the same prompt will return the same brand list. The chance they also come back in the same order is closer to one in a thousand.
That means an audit tool reporting that you “rank third in ChatGPT” is not telling you where you stand. It is telling you where you happened to appear on one specific occasion that is extremely unlikely to repeat.
What does stabilise with enough runs is frequency. How often your brand appears across many runs of the same prompt gives a usable visibility figure. SparkToro recommends at least 60 to 100 runs per prompt. Rank position from a single run does not.
An audit that doesn’t tell you how many runs it is based on is not really an audit.
For the same reason, every measurement is a snapshot. The AI tools are updated continuously, so measure again at least once a quarter and after major changes to the website.
Three questions to ask before you buy an AI audit
Whoever you work with to measure your AI visibility, there are three things you should get a clear answer to.
How is the data collected? API calls, logged-in interface scraping, logged-out interface scraping, and real-user panels are four fundamentally different answers with fundamentally different levels of reliability.
How many runs per prompt, and over what time period? Ten runs gives a rough picture. Sixty to a hundred gives a stable measure. One run tells you very little.
Are the prompts run in your language, from a relevant location? ChatGPT automatically rewrites queries based on your geographic location. An audit run from a server on the other side of the world using generic English prompts is not measuring what your local customers actually see.
A provider who cannot answer all three questions clearly either has no solid methodology, or no visibility into the one they are using.
How we do it at Monprez
We scrape the interface your customers actually see, not the API behind it. We run prompts in multiple rounds, in the language and geographic context relevant to your business. We separate mentions — meaning your name appearing somewhere in a response — from citations — meaning ChatGPT or Perplexity actually linking to your website as a source. That distinction matters.
And we tell you how we did it, so you can compare and decide for yourself.
Take the three questions to whoever you hire, us included. If you order our AI visibility audit, you get the answers in writing in the report, along with how often your business is mentioned compared with two or three competitors. If you want to see where you stand first, the test in our other article takes ten minutes, and our guide to GEO explains what decides whether you get mentioned at all.