How Accurate Are AI Visibility Tools? We Checked Ours for 28 Days

Dashboard panel showing ChatGPT naming a business in 82 percent of answers over 28 days, with a range of 64 to 92 percent
Explore with AI
Read with ChatGPTRead with ClaudeRead with AI Mode

Exact on the Record, Approximate on the Rate

AI visibility tools are accurate about what an assistant said and approximate about how often it says it. The stored answer, its date, and the sources it cited are a record, and a record is either right or wrong. The mention rate built from those answers is an estimate with a range around it. Anything modeled on top, like prompt volume or a blended score, is softer still.

That's three kinds of number sharing one dashboard, and most arguments about accuracy come from reading one kind as another. At least one vendor says so in writing. Scrunch's help center note on prompt volume, updated in September 2026, tells its own customers that AI visibility data should be treated as probabilistic and directional, not absolute.

So we ran the test on ourselves. Below is one tracked question, asked on ChatGPT, Gemini, Claude, Perplexity, and Google AI once a day for 28 days, with every answer kept. It shows where a tracker's numbers hold and where they wobble, ours included.

One Question, 28 Days, 119 Answers From Our Own Tracker

The business is a Miami mobile car wash that follows its prompts in Lighthouse Local, the same one behind our guide to AI visibility for local business. The question is one a customer would type, asking for the best mobile car wash in the city. From October 8 to November 4, 2026, it went to each assistant once a day.

AssistantAnswers on recordNamed the business28-day rateWeek by week
ChatGPT282382%6 of 7, 6 of 7, 5 of 7, 6 of 7
Gemini24313%2 of 5, 1 of 5, 0 of 7, 0 of 7
Claude2800%0 of 7 every week
Perplexity28829%2 of 7, 0 of 7, 4 of 7, 2 of 7
Google AI1119%1 of 4, 0 of 2, 0 of 2, 0 of 3
All combined1193529%

Not every day produced an answer on every assistant. Gemini has 24 and Google AI has 11. Google doesn't show an AI Overview on every search, and a tracker reading the live results page can only record one that appears. A day without an answer isn't a miss, so those two rates are counted over 24 and 11 answers, not 28. That's the first accuracy check for any tool. Ask what the rate is divided by.

Now read Perplexity week by week. It named the car wash in 2 of 7 answers, then 0 of 7, then 4 of 7, then 2 of 7. Taken alone, week two says Perplexity dropped the business and week three says it came roaring back. The 28-day count, 8 of 28, says neither thing happened.

Perplexity named one car wash in 2, 0, 4, then 2 of 7 weekly answers while the 28-day rate held at 8 of 28

ChatGPT tells the same story from the other side. It named the business on 23 of 28 days, and on five scattered days it didn't. An owner who checked by hand on October 13, 19, 27, 28, or 30 would have walked away sure the business was invisible on the assistant that names it most.

The Record Is Exact, and the Thing It Records Keeps Moving

Start with the layer you can verify with your own eyes. A tracker that stores each answer verbatim, along with its date and its cited pages, has given you something checkable. Open the answer, read it, and the question of whether you were named that day is settled.

What the record shows is how much the answer itself moves. Across ChatGPT's 28 answers to that one question, 22 different business names appeared. The lists ran from four names to eight. No business made all 28, eleven names showed up exactly once, and the list matched the previous day's in full on just 3 of 27 days.

That churn isn't a quirk of one car wash. A September 2026 preprint, a finite-sample audit of repeated LLM queries, collected 4,500 responses to 50 buying questions. In its February wave, a single call typically recovered between 80% and 92.5% of the brands seen across five calls. One answer leaves part of the field out.

The cited sources churn as well. SISTRIX's citation drift analysis followed 82,619 prompts for 17 weeks into April 2026 and found ChatGPT replacing 74% of its cited sources from one week to the next, with Google's AI Mode replacing 56%. A tool can be exactly right about Tuesday and out of date by the following Tuesday. It's why following what ChatGPT says about your business means keeping every answer instead of the latest one alone.

A Mention Rate Is an Estimate, and the Answer Count Sets Its Range

A mention rate is one count divided by another, and a small count carries a wide range. ChatGPT's 23 of 28 is 82%, and with only 28 answers the honest range around it runs from about 64% to 92%. Perplexity's 8 of 28 is 29%, which could sit anywhere between 15% and 47%. Those are standard 95% intervals, the same arithmetic a pollster uses.

Seven answers are far worse. At a true rate near 29%, a week of daily checks can land roughly 34 points to either side, which is enough on its own to produce weeks that read 29%, 0%, 57%, and 29%. The range closes as answers pile up. At that same rate it's about 17 points either way at 28 answers, 9 at 100, and 4 at 400.

The range around a 29 percent mention rate narrows from about 34 points at 7 answers to about 4 points at 400

Even a clean zero has a range. Claude went 0 for 28, which puts its real rate anywhere from nothing to about 12%, and on November 5, the day after this window closed, Claude named the car wash.

The research agrees on the size of the problem. Another September 2026 preprint, the Dice Roll Method, reanalyzed about 190,000 observations from five brand-recommendation audits. In its simulations, a single comparison at five runs had a 15% to 19% chance of detecting a large effect, and it took 15 to 25 runs to reach 80%.

The same paper carries the good news. Measured across 250 different questions, reliability reached 0.96 at five runs each. Breadth steadies a number faster than depth does. That suggests a rate built from every question in an account settles sooner than the rate on any one of them. So read one question on one assistant over a month, and read the whole account sooner.

The API and the App Read the Web Differently

The next question is whether the tool sees what your customer sees. Many trackers, ours included, ask through each AI company's developer channel. Lighthouse Local's AI visibility tracker uses it for four of the assistants it follows, with search turned on.

That channel isn't the app on a customer's phone. In February 2026, Genezio published a comparison of the GPT-5.2 API and chatgpt.com built on 3,645 conversations about UK banking over 27 days. Of 3,856 distinct searches the model ran, not one appeared on both paths, and of 5,701 source pages only 704, or 12.3%, were used by both. It's one industry in one country, run by a vendor, so treat the exact figures as a single data point. The direction is the part to keep.

Personal memory and location widen the gap further, which is a big part of why AI recommendations differ by user. Underneath all of it, the model itself keeps changing. OpenAI's list of retired API models says it regularly retires older ones and gives each date.

None of this makes the number useless. A tracker that asks the same way every day is consistent with itself, so its changes are comparable over time once enough answers pile up, even when its level differs from one person's screen. Compare the tool with its own history, never with your phone.

Modeled Numbers Sit Furthest From the Evidence

Everything so far rests on captured answers. Some dashboard numbers don't. Prompt volume is the clearest case. The Scrunch note quoted earlier explains that third-party tools lack the platforms' own data, so they start from small browser-extension panels and scale up, and it says outright that Scrunch doesn't treat prompt volume as a precise demand metric.

Blended scores and sentiment grades belong here too. Each is a formula applied to the answers, and two vendors rarely share a formula. In May 2026 an agency chief executive told Digiday's report on marketer skepticism that three tools given the same prompts return three different answers. That's less a scandal than a reminder to ask what each number is made of.

Number on the dashboardWhat it rests onHow to read it
The stored answer and its dateA captured recordExact. Open it and check
Named or not in one answerThat recordCheckable. Open the answer and read it
Sources cited in one answerThat recordExact for that day, with heavy weekly churn
Mention rate, one question on one assistant7 to 30 answersWide. Roughly 15 points or more either way at 28 answers
Mention rate across the whole accountHundreds of answersNarrower, so month-to-month changes read better
Week-over-week change on one questionTwo samples of sevenMostly noise
Prompt volumePanel estimatesDirectional, not a demand figure
Blended visibility scoreA vendor formulaAsk for the formula first

Those rows double as questions for a sales call, and our guide on how to evaluate an AI visibility tracking tool turns them into a vendor checklist.

Test Any Tracker in 20 Minutes

You don't have to take a vendor's word, or ours. Pick one tracked question and run these six checks.

  1. Open the evidence. Pull the stored answers for the last seven days. If the tool can't show the full text and date of each one, nothing else it reports can be checked.
  2. Recount the rate. Count the days you were named and divide by the answers on record. It should match the dashboard to the decimal. This part is arithmetic.
  3. Find the blanks. Look for days with no answer and confirm they're left out of the rate, not counted against you.
  4. Click two sources. Follow two cited links from one answer. The pages should exist and say what the answer says they say.
  5. Ask it yourself five times. Use a fresh chat with memory off, and log out if the assistant lets you. Your five answers won't match the tool's on any given day. Over a few weeks, high should stay high and low should stay low.
  6. Refuse the weekly verdict. Read a single question's rate over four weeks or more, and the account-wide rate month to month.

Keep the result somewhere you'll see it again. Here's a scorecard to copy.

AI visibility tracker accuracy check
Tool ______   Question ______   Date ______

1. Full answer text and date shown for 7 days      yes / no
2. My recount ___ of ___   Dashboard says ___%      match / no match
3. Days with no answer ___   Left out of the rate   yes / no
4. Two cited sources opened, both real and relevant yes / no
5. My 5 manual asks, named ___ of 5                 same direction / opposite
6. I read the rate over 4 weeks or more             yes / no

Trust the Record, Read the Rate With Its Range

So how accurate are AI visibility tools? On the record, exactly as accurate as you can verify, which is why the stored answer matters more than any score. On the rate, accurate to within a range that shrinks as answers pile up. On matching one customer's screen, only roughly, which is true of any tracker that asks through a developer channel.

That's how we'd ask you to read ours. Lighthouse Local for Business keeps every answer from each assistant with its date and the pages it cited, so any rate on the dashboard can be traced back to the answers behind it. When the answers point to fixes, our specialists can take that work on for you.

A tool that shows you its evidence has earned a hearing. Trust the record, read the rate with its range, and let a month pass before you call anything a change.

Frequently asked questions

How accurate are AI visibility tools?

They're exact about what an assistant said on a given day and approximate about how often it says it. A stored answer is a record you can open and check. A mention rate built from 28 answers can sit roughly 15 points to either side of the number shown, and modeled figures like prompt volume are estimates.

Why does my AI visibility tool show something different from what I see in ChatGPT?

Three things differ. The assistant gives a different answer each time it's asked, the app personalizes with memory and location, and many trackers ask through the developer channel, which searches the web differently from the app. A 2026 Genezio study of 3,645 conversations found only 12.3% of source pages were shared between the two paths.

How many answers does a mention rate need before I can trust it?

More than a week's worth. Seven daily answers can swing about 34 points either way at a rate near 30%. Around 28 answers on one question narrows that to roughly 17 points, and about 400 answers bring it under 5.

Are AI visibility scores reliable?

A blended score is a vendor's formula applied to captured answers, so it's only as reliable as the answers behind it and the weights in the formula. Ask how it's calculated, how many answers it's built on, and whether you can open those answers yourself.

Can two AI visibility tools give different numbers for the same business?

Yes, and it doesn't mean either one is broken. They ask different questions, on different days, through different channels, a different number of times, and they score the results with different formulas. Compare each tool with its own history, not with another tool.

How do I test whether my tracker is accurate?

Open the stored answers for one question, recount the rate by hand, check that blank days are left out, click two cited sources, and ask the same question yourself five times in a fresh chat. If the evidence is visible and your recount matches, the record is sound.

Mentioned in this guide

How It WorksFor BusinessPricing