How to measure if your AI visibility is really improving (and not just noise)

To measure whether your AI visibility is improving, ask the same fixed buyer questions in the same AI apps under the same conditions, several times each, and compare the share of answers that name you with control questions you did not work on. Believe a change only when it beats the noise margin twice in a row.

Back to Blog

This guide to measuring the results of answer engine and generative engine optimization (GEO), updated November 2026, turns that rule into a routine you can run with ChatGPT, Gemini, a spreadsheet, Google Search Console and GA4.

Far & Wide is an AI visibility platform that asks the same buyer questions in the AI apps you pick every 7 days, several times each, and has agents make the site changes you approve; below is the by-hand version, with its real hours.

Key points

  • One answer proves nothing. Count the share of many answers that name you, never a single reply or screenshot.
  • Sources churn weekly. SISTRIX measured 74% of the domains ChatGPT cites are new every week, and 56% in Google AI Mode.
  • Pool before you judge. With 180 answers a month per app, the target rate alone carries a margin of about 8 to 10 points; compared against 5 control questions, a gain must beat about 17 to 21.
  • By hand costs €0 and 19 hours 15 minutes in month one, then 17 hours a month, before any change is made to your site.

How do I measure if my AI visibility is improving?

You measure whether your AI visibility is improving by tracking your mention rate (the share of AI answers that name your brand) on a fixed question list, and comparing how far it moved on the questions you worked on with how far it moved on control questions. AI visibility is the percentage of AI responses in which a brand appears, so it only exists as a count over many answers.

What is the difference between monitoring and measuring improvement?

Monitoring (or tracking) records what AI says about you on a schedule: same questions, same conditions, every week, giving a time series. Measuring improvement tests whether a dated change moved the questions it targeted more than the control questions moved, by more than the noise margin. Monitoring tells you what happened; measuring tells you whether your work caused it. For tracking basics, see what AI visibility tracking is.

Which questions should you track, and which are controls?

Write 15 target questions and 5 control questions, and freeze the wording. Target questions are buyer questions you plan to work on, such as “best invoicing app for freelancers in Germany”. Control questions are real buyer questions you will not touch, so they show whether the whole market moved. Your real queries in Search Console are a good source; see finding the prompts your customers use.

Keep questions that name your brand apart. “Is [Brand] any good?” returns your name almost every time and inflates the rate.

Define a hit before you start, for example “named anywhere” or “named in the first three”, and never change it mid-series.

What should stay the same between checks?

Hold seven conditions fixed, because each one changes the answer on its own:

  1. Exact wording, pasted from the sheet.
  2. The same app. The ChatGPT app is not the API, and Google AI Overviews is not Google AI Mode; real apps and APIs differ.
  3. Country and language, with no VPN switching.
  4. Account state: logged out, or one test account with memory off.
  5. Persona: if you add “I run a 10-person agency”, use that line every time.
  6. A fresh chat for every ask.
  7. The same weekday, so weekly cycles do not pose as trends.

How many times should you ask each question?

Ask each question at least 3 times per app per check, in fresh chats, and judge the pooled monthly result rather than any one question. Answers differ because the model samples its wording, sometimes searches the web and sometimes answers from memory. In SparkToro’s study, where 600 volunteers ran 12 prompts 2,961 times, the advice was to ask “at least 60-100X” and average; we cover what that means when buying a tool in what counts as proof from a tracker.

Three asks a week for four weeks gives 12 answers per question per app a month, far below 60, so one question’s trend is mostly luck. Pool all 15 target questions into one number.

Cited sources move too. SISTRIX tracked 82,619 prompts over 17 weeks: 74% of ChatGPT’s cited domains were new each week, while in Google AI Overviews 53% of prompts kept every source for all 17 weeks. Expect ChatGPT’s cited sources to bounce more.

What should you log for every answer?

Log one row per answer, with the full answer pasted in, because a score hides wrong claims and tone.

ColumnWhat to recordA real move looks like
NamedYes or no, by your hit definitionMention rate beats the margin
PlaceFirst, second, laterAverage place improves two months running
CompetitorsEvery brand listedYour share of voice rises
Cited sitesEvery domain linkedYour page recurs across rounds
TonePositive, neutral, negativeFewer negative mentions
Full answerWord for wordWrong claims disappear

How do I tie a change to the questions it should move?

Keep a dated change log, one row per change, naming the questions it should move before you ship it. A worked example:

FieldEntry
Shipped3 November 2026
ChangePublished “[Brand] vs [Competitor]” and “[Competitor] alternatives” pages
Should move“alternatives to [Competitor] for small teams”, “[Competitor] or [Brand]?”, “cheaper option than [Competitor]”
Controls5 untouched questions
BaselineOctober mention rate per app, targets and controls
First checkSearch Console URL Inspection shows both pages indexed
VerdictEnd of November, confirmed end of December

A new page like these takes 3 to 4 hours by hand; Far & Wide’s Content agent writes comparison pages for your OK. Check indexing first, so “no move” does not just mean “not crawled yet”, and watch whether the new pages start appearing among cited sites. Three questions give only 36 answers a month per app, a ±18 to 23 point margin, so judge them inside the pooled target group rather than on their own.

When is a change big enough to believe?

Believe a change when the target questions move more than the control questions by more than the margin below, in the same direction on two monthly checks in a row. The margins come from the standard margin of error formula, applied to a change between two checks of n answers each (about 1.4 ÷ √n at worst):

Answers per check (n)At 20% mention rateAt 50%How you get n
15±29 points±36 points15 questions, asked once
45±17 points±21 points15 questions × 3 asks, one app, one week
60±14 points±18 points5 controls × 3 asks × 4 weeks
180±8 points±10 points15 questions × 3 asks × 4 weeks, one app
750±4 points±5 points50 questions × 3 personas × 5 asks, one app (arithmetic only: repeat asks of one question overlap, so the real margin is wider)

Worked example. In October, ChatGPT named you in 36 of 180 target answers (20%) and 12 of 60 control answers (20%). In November, after a month of logged changes, targets reached 81 of 180 (45%, up 25 points) and controls 14 of 60 (23%, up 3). The net gain is 22 points; combining both margins gives about ±17, so 22 clears it. If December holds, believe it. A rise from 20% to 27% with flat controls would sit inside the ±8 to 10 point margin: noise.

These margins are best-case, because answers to one question are not fully independent. Treat them as a floor.

How do I track whether AI is recommending me over time?

Collect answers every week and judge them every month: run the same list in the same apps weekly, pool four weeks into one mention rate per app, and compare month to month against controls. Report each app separately; Pew found in 2026 that 44% of U.S. adults have ever used ChatGPT, 24% Gemini and 17% Copilot, so a blended number hides where you gained or lost.

See your mention rate for yourself in ChatGPT and Gemini

  1. Pick one target and one control question from your sheet.
  2. Open the ChatGPT app in your fixed account state, memory off or in a temporary chat.
  3. Paste the question into a new chat and note whether ChatGPT searched the web (sources shown) or answered from memory.
  4. Log named, place, competitors, cited sites and the full answer.
  5. Repeat in two more fresh chats, then do the same in the Gemini app.
  6. Repeat on the same weekday next week and read the result as a monthly rate.

A one-off first check shows where you start; repeating it under fixed conditions turns it into a trend.

Does better AI visibility reach your traffic and pipeline?

Check three places, because a mention that matters shows up as visits, branded searches and buyers who say where they heard of you. G2’s 2026 survey of 1,076 B2B decision makers found 51% of software buyers start research with an AI chatbot more often than Google, and 69% said AI chatbots surfaced information that led them to choose a different vendor than expected. Forrester reported that 94% of business buyers use AI in buying.

  1. GA4. In Reports, Acquisition, Traffic acquisition, filter the session default channel group to AI Assistant, which covers visits from sources like ChatGPT, Gemini and Copilot. Google’s own definition excludes AI Overviews and AI Mode, whose clicks count as Organic Search. The number is a floor: visits without a referrer, from apps or copied links, still land in Direct.
  2. Google Search Console. Track clicks on queries containing your brand, same weeks month to month. Since 31 August 2026 every site also gets generative AI performance reports showing how often its URLs appeared in AI Overviews and AI Mode.
  3. “How did you hear about us?” Add ChatGPT, Gemini and “another AI assistant” to the form, and ask on sales calls.

Read all three next to your change log, since branded searches rise for many reasons. Measuring this by hand costs €0 in cash but 19 hours 15 minutes in month one and 17 hours a month after; set those hours against any pipeline the three signals show. For revenue maths, see AEO ROI measurement.

Checking AI mentions by hand every week versus automated tracking: which works better?

Checking by hand gives a usable monthly trend for €0 in cash but 19 hours 15 minutes in month one and 17 hours a month after, with 12 answers per question per app against the 60 to 100 SparkToro advises, and no change made to your site. Tracking-only tools add volume but leave every fix to you. A platform that measures and also makes the changes is the option that does both jobs, and Far & Wide’s Audit and Fix plans are that kind.

The by-hand routine, defined once:

  • Set-up, once: 2 hours 15 minutes. 15 target and 5 control questions (45 min), the fixed-conditions sheet (15 min), the spreadsheet (30 min), GA4 and Search Console baselines plus the change log (30 min), and AI options added to your “how did you hear about us” form (15 min).
  • Weekly round: 4 hours. 20 questions × 2 apps × 3 asks = 120 answers, at 2 minutes each to open a chat, ask, read the answer and its sources, and log it.
  • Monthly review: 1 hour. Target and control rates per app, cited sources, GA4, Search Console, “how did you hear about us” answers, change log.
  • Totals: month one = 2 h 15 min + 4 × 4 h + 1 h = 19 hours 15 minutes; every later month = 4 × 4 h + 1 h = 17 hours.

Those hours cover measuring only. The change-log row and the indexing check for each change are counted with the change, not in these measuring hours. A rewritten page adds 1 to 2 hours and a new page 3 to 4 hours, code changes need someone who can edit the site, and a steady monthly output of pages and outreach is extra work again. The routine also asks each question 3 times a week, far fewer than the 60 to 100 asks per question recommended in SparkToro’s January 2026 study of 2,961 runs.

By hand (spreadsheet)Tracking-only toolFar & Wide Audit and Fix (measures and makes changes)
Cash cost€0A subscriptionA subscription
Your time19 h 15 min month one, 17 h a month after, before any changeReading reports, plus every changeApproving finished changes, a few minutes a week
AI answers a month480 (20 questions × 2 apps × 3 asks × 4 weeks)Varies by toolUp to 12,000 on Audit and Fix (50 questions × 4 apps × 3 personas × 5 asks × 4 times a month); Google AI Overviews gives one answer however often it is asked, so the real count is a little lower
Changes on your siteYou make themYou make themAgents prepare code, page fixes and new pages; after your OK, published on a connected site (with an undo) or handed over as ready-to-paste code, and checked live; outreach ready to send, sent after your OK, placement never guaranteed
After a changeRe-measure if you rememberYours to connectThe questions a fix targeted are measured again at the next audit

Far & Wide asks the same 50 questions in 4 of 6 apps you pick (ChatGPT, Gemini, Perplexity, Microsoft Copilot, Google AI Overviews, Google AI Mode), in the real apps, from your buyers’ country and language, with questions that name your brand counted apart. Google AI Overviews gives one answer however often it is asked, so its counts are lower.

What measuring setup fits your team size?

TeamBy hand: setupBy hand: time a monthWith a platform that makes the changes
1 marketer20 questions × ChatGPT + Gemini × 3 asks, spreadsheet, GA4, Search Console19 h 15 min month one, 17 h a month afterAudit: the same 50 questions in the 4 apps you pick, 3 personas × 5 asks every 7 days; one try-out fix a month in each area, published after your OK, checked live, re-measured at the next audit
2 to 5 peopleSame, plus Perplexity or Microsoft Copilot: 20 × 3 apps × 3 asks = 180 answers = 6 h a round27 h 15 min month one, 25 h a month after (each extra app adds 8 h a month)Fix: the same measuring, plus a large number of fixes every month, each published after your OK, checked live and re-measured
5+ people or several markets50 questions × 4 apps (adding Google AI Overviews or AI Mode) × 3 asks = 600 answers = 20 h a round80 h a month in weekly rounds alone, before set-up and review; with 3 personas × 5 asks, 3,000 answers = 100 h a round, 400 h a month in rounds aloneFix, or Enterprise: everything in Fix, with volumes tailored to you

On every row, the team’s time with Far & Wide is approving finished changes, a few minutes a week; tracking-only tools cut the asking but leave every change to the team. Even for one person, 17 hours a month is over two working days (at 8 hours a day) spent on measuring alone, before any change is made. A platform that also ships the changes takes both the measuring and the work off the list.

Common mistakes

  • Comparing screenshots: one ask on 15 questions carries a ±29 to 36 point margin.
  • Skipping controls, so a market-wide shift looks like your win.
  • Declaring victory after one month instead of two in a row.

Quick checklist

  • 15 target + 5 control questions, wording frozen
  • Seven conditions held fixed
  • 3 asks per question per app, fresh chat each time
  • Full answer, place, competitors, cited sites and tone logged
  • Every change dated, with the questions it should move
  • Monthly target-minus-control compared with the margin table
  • GA4, branded Search Console queries and “how did you hear about us” reviewed monthly

FAQ

Open the ChatGPT app in a fixed account state with memory off, and ask a buyer question that does not name you, 3 times in 3 fresh chats. Count how many answers name you, at what place, and which sites are cited. Repeat on the same weekday each week, do the same in Gemini, and compare monthly rates, not single answers.

Keep measuring, and get the changes made

Hand-checking 20 questions takes 19 hours 15 minutes in month one and 17 hours a month after, and leaves every change to you. On Audit and Fix, Far & Wide asks your 50 buyer questions 5 times as 3 buyer personas every 7 days in the 4 AI apps you pick, keeps every answer word for word, and its Technical, Web design, Content and Authority agents prepare the fixes: on Audit, one try-out fix a month in each area, on your most urgent problem; on Fix, a large number of fixes every month. You see each fix first; after your OK it is published on your connected site (with an undo) or handed over as ready-to-paste code, and checked live, and the questions it was meant to move are measured again at the next audit.

Audit plan, no card needed. If you let the trial end, nothing is charged and your results stay in your account.

Start 7-day free trial