How to measure your visibility in AI search — and why every dashboard undercounts it

You cannot manage what you cannot see, and AI search has quietly broken the thing marketers were watching. The engine reads your content thousands of times for every visitor it sends back, so a dashboard built on clicks will tell you AI is worthless while it is busy shaping the answer. Measuring it means watching five layers, not one.

For a decade the scoreboard was simple: a keyword had a rank, a rank produced clicks, and clicks showed up in one report. Answer engines collapse that chain. The model rewrites your page into an answer, names a few sources, and most readers never click. The visit you used to count is now the exception — which means the absence of a click is no longer the absence of influence. This piece lays out what to measure instead, grounded in what the platforms themselves publish, and closes with the one mental model that holds it together: the AI Visibility Measurement Stack.

How is AI-search visibility different from a keyword ranking?

A rank is a single number. AI-search visibility is a funnel, and each stage fails differently.

01

CrawlMeasure with · Server logs · AI Crawl Control

Are the right bots fetching you — and the right kind?

02

IndexMeasure with · Coverage reports · proxy: do you get cited at all

Are you in the candidate set a query can retrieve from?

03

CitationMeasure with · Bing AI Performance · answer-tracking

Are you named as a source inside the generated answer?

04

ReferralMeasure with · GA4 referral segmentation

Does the citation send a person to your site?

05

ConversionMeasure with · GA4 conversions, segmented by AI source

Does AI-referred traffic do anything once it arrives?

Most teams instrument only layer 04 — referral traffic — because it is the one their analytics already counts. The other four are where the visibility is won or lost, and no single dashboard spans them. You stitch the stack together yourself.

Framework: Martech LLC · instruments map to the primary sources cited inline

The layers are not interchangeable. A page can be crawled but never retrieved; retrieved but never cited; cited but never clicked; clicked but never converted. Each break has a different cause and a different instrument, and the tooling for them lives in four different places — your server logs, Bing's webmaster console, Google's Search Console, and your own analytics. The work is not finding one perfect dashboard. It is accepting that one does not exist yet and stitching the view yourself.

To make that concrete, take one real page and walk it down the stack — the question to ask at each layer and the instrument that answers it:

The page you want to measure

A pricing-comparison page you just published — and the question “is it winning AI search?”

The five layers you have to instrument

1. 01 · Crawl

   Are the AI bots fetching this page — and which ones?

   tests · is the engine even reading you

   The instrument at this layer

   Server logs / AI Crawl Control: watch for OAI-SearchBot, PerplexityBot, Googlebot.

2. 02 · Index

   Are you in a candidate set a query can retrieve from?

   tests · could you be surfaced at all

   The instrument at this layer

   Proxy it — probe your buyers' questions and check whether you're cited anywhere.

3. 03 · Citation

   Are you named as a source inside the generated answer?

   tests · did you actually get cited

   The instrument at this layer

   Bing AI Performance report + answer-tracking across the engines that show sources.

4. 04 · Referral

   Does the citation send a person to the page?

   tests · does the visibility move a human

   The instrument at this layer

   GA4 referral segmentation by AI source — the one layer most teams already count.

5. 05 · Conversion

   Does that AI-referred visitor do anything once they arrive?

   tests · is the traffic worth anything

   The instrument at this layer

   GA4 conversions segmented by AI source, not blended into the channel total.

No single dashboard spans these five — so you stitch them. Most teams measure only layer 04 because their analytics already counts it, and conclude AI search does nothing. The visibility was won or lost two layers earlier, in the crawl and the citation, where they never looked.

Stages map to the AI Visibility Measurement Stack · instruments cited inline in the prose

Why does referral traffic undercount your AI visibility?

Because the click is the smallest thing the engine does with your content. It reads you constantly; it refers you rarely.

Cloudflare, which sits in front of a large share of the web, measured the gap directly: in July 2025 the crawl-to-referral ratio reached roughly 38,065 pages crawled per referred visit for Anthropic, against 1,091 for OpenAI, 195 for Perplexity, 41 for Microsoft, and 5.4 for Google.[9] Even the most efficient engine took your content five times for every reader it returned; the least sent one visitor for every thirty-eight thousand reads.

Anthropic38,065 : 1

OpenAI1,091 : 1

Perplexity195 : 1

Microsoft41 : 1

Google5.4 : 1

Read that the right way: a click-based KPI will always make AI search look worthless, because the engine consumes your content far more than it forwards a reader. The crawl is the engagement. Referral is the leftover.

Source: Cloudflare, “The crawl before the fall of referrals” / “crawl-to-click gap,” July–Aug 2025 · ratios = pages crawled per referred visit · bars log-scaled

And most of that reading is not even for live answers. Cloudflare reported that over the prior twelve months, 80% of AI crawling was for training, against 18% for search and just 2% for user actions.[10] The economics of this imbalance are now explicit. On July 1, 2025 Cloudflare announced it had become the first internet infrastructure provider to block AI crawlers by default, asking every new domain whether to allow them, and launched a Pay Per Crawl model letting owners charge for access.[11]

If your only KPI is the click, you have chosen the one number that makes AI search look like it does nothing — while it reads everything.

The lesson is not that referral traffic is useless. It is that referral is a lagging, lossy signal of a much larger influence, and it must be read alongside the crawl and the citation, never alone.

Which crawler do you actually need to let in?

Here is where measurement turns into a control problem, and where teams quietly delete themselves from AI answers. Every engine runs more than one bot, and they do different jobs.

The clean line is training versus live retrieval. OpenAI documents that GPTBot crawls content that may be used in training its foundation models[6] — blocking it is a training opt-out, nothing more. The bot that governs whether you appear in answers is a different one entirely:

Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.

That sentence is from OpenAI's own crawler documentation, and it is the whole game.[5] Block GPTBot and you stay in ChatGPT's search answers; block OAI-SearchBot and you disappear from them. The same split runs through the other engines. Anthropic documents that disabling its Claude-User agent prevents its system from retrieving your content in response to a user query, which may reduce your visibility for user-directed web search.[7] Google keeps its AI-training control fully separate from search: it states that Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal.[8]

So the first thing to measure is your own robots.txt and bot-management layer, read against your server logs. If a retrieval bot is being blocked — by a rule, a firewall, or a default — no downstream metric will ever recover, because the engine was never allowed to read the page it would have cited.

OpenAI

Training · safe to block

GPTBot

Retrieval · never block

OAI-SearchBot

Block → gone from ChatGPT search answers

Anthropic

Training · safe to block

ClaudeBot

Retrieval · never block

Claude-User

Block → reduced visibility in user web search

Google

Training · safe to block

Google-Extended

Retrieval · never block

Googlebot

Block → out of Search, AI Overviews & AI Mode

Perplexity

Training · safe to block

—

Retrieval · never block

PerplexityBot

Block → not retrieved for Perplexity answers

Blocking a training crawler is a choice with no effect on whether you are cited. Blocking a retrieval crawler — often by an over-broad firewall rule or a default — is an accident that erases you from the answer, and no downstream metric can recover it. Audit this first.

Sources: OpenAI / Anthropic / Google crawler docs · block-effects per each engine's own documentation

Where can you actually see your citations?

Once the right bots are in, the question becomes whether you are named in the answer. The reporting here is wildly uneven.

Bing / CopilotCopilot, Bing AI answers

AI Performance report — citations, cited pages, grounding queries. No clicks.

Native report

GoogleAI Overviews, AI Mode

Folded into Search Console’s “Web” type — no AI-specific breakout or filter.

Blended in

GeminiGemini app answers

No publisher console — grounding draws on Google’s index; not separately reported.

No console

ChatGPTChatGPT search answers

No publisher console — measurable only via server logs (OAI-SearchBot) + GA4 referral.

No console

PerplexityPerplexity answers

No publisher console — measurable only via server logs (PerplexityBot) + GA4 referral.

No console

There is no unified report. Until there is, AI-visibility measurement is a stitched methodology: server-log bot analysis for the crawl, Bing AI Performance for citations, Search Console’s “Web” type for the blended Google surface, and GA4 referral segmentation for the traffic that survives.

Sources: Bing Webmaster Tools (Feb 2026) · Google Search Central “AI features” (Dec 2025) · OpenAI & Perplexity crawler docs

One engine treats citation as a first-class metric. In February 2026, Bing Webmaster Tools launched a public-preview AI Performance report that measures how often your content is cited as a source inside AI answers — exposing Total Citations, Average Cited Pages, Grounding Queries, and page-level citation activity.[1] Bing is blunt about why it built it: visibility, the announcement says, is "not only about blue links" but "about whether your content is cited and referenced when AI systems generate answers."[2] Note the deliberate framing: it counts citations, not clicks.

Google goes the other way. Its Search Central guidance states that sites appearing in AI features such as AI Overviews and AI Mode are folded into overall search traffic in Search Console and reported under the "Web" search type — with no separate AI breakout.[3] The counting rules are the ordinary ones: standard impression rules apply, and clicking a link to an external page in AI Mode counts as a click.[4] You can see the blended total; you cannot isolate the AI slice. For ChatGPT, Perplexity, and Gemini there is no publisher console at all — the only native evidence you have is your own server logs showing OAI-SearchBot and PerplexityBot fetching the page.

How do you measure the click that did happen — and the one that didn't?

Two numbers matter at the bottom of the funnel: how far the click collapsed, and what the surviving traffic does.

The collapse is now measured, not guessed. The Pew Research Center tracked real browsing behavior across 68,879 searches from 900 U.S. adults and found that when a Google AI summary appeared, users clicked a traditional result in just 8% of visits — against 15% when no summary appeared.[12] Half the clicks, gone, on the queries that trigger a summary.

No AI summary on the page15% click a result

AI summary present8% click a result

Roughly half the clicks disappear on the queries that trigger a summary — so a click-based KPI does not just undercount AI search, it actively reports a loss on exactly the surfaces where the engine is working hardest. The click that did not happen is not an absence of influence.

Source: Pew Research Center (Jul 2025) — 68,879 searches across 900 U.S. adults · figures cited inline

What survives behaves differently, and you can isolate it. AI engines arrive in your analytics as referral hostnames — chatgpt.com, perplexity.ai, gemini.google.com, copilot.microsoft.com — so a GA4 segment or channel grouping built on those referrers turns "AI traffic" into a number you can report and trend. The traffic is worth isolating because it is growing and it is good: Adobe Analytics measured a 1,200% jump in U.S. retail traffic from generative-AI sources between July 2024 and February 2025, with AI-referred visitors showing a 23% lower bounce rate — though still 9% less likely to convert than other visitors.[13] More engaged, not yet closing as well: exactly the kind of nuance a blended report hides and a segment reveals.

What this approach does not prove

Honesty about the gaps is part of the method, because the measurement layer is young and several of its inputs are contested.

The crawl-to-referral ratios are point-in-time snapshots that move fast — Anthropic's figure ranged from roughly 38,000:1 to near 50,000:1 within weeks of the same summer — so cite the exact window, never a round "always." Cloudflare is also an interested party: it sells the crawler-blocking and Pay Per Crawl products its data argues for, even though the underlying network measurements are first-party and credible. The Pew click figures were publicly disputed by Google, which argued the query sample skewed toward low-engagement searches; Pew's measured numbers stand, but they describe one month of one panel. The bot-purpose lines are the operators' own documentation, not independently audited behavior — a distinction that matters most for Perplexity, which Cloudflare has separately accused of using undeclared crawlers to evade no-crawl directives.

And one popular "fix" earns no place in the stack. As of April 2025, Google's John Mueller said none of the AI services had said they use llms.txt, comparing it to the deprecated keywords meta tag.[14] Until an engine confirms it consumes the file, treat it as housekeeping, not a measurement or visibility lever.

What this means for the work

Four shifts follow directly, in order of effort-to-impact.

Audit your bots before you audit your content. Read robots.txt and your bot-management rules against server logs, and confirm the retrieval crawlers — OAI-SearchBot, PerplexityBot, Claude-User — are allowed. Blocking a training crawler is a choice; blocking a retrieval crawler is an accident that erases you from answers.

Add citation as a tracked metric, not just clicks. Where a native report exists (Bing AI Performance), watch it. Where it does not (Google, ChatGPT, Perplexity, Gemini), approximate it — server-log fetch frequency for the crawl, and periodic answer-checking for the citation.

Segment AI referral traffic in GA4 today. Build the referrer segment now so you have a baseline before the volume grows. A 1,200% rise is only visible if you were counting before it started.

Report the stack, not the click. Present crawl, citation, referral, and conversion together, with the caveat that referral undercounts influence. A leadership team that sees only the click will defund the channel that is quietly winning the answer.

The single mental model is the stack itself: five layers, four instruments, one honest caveat that no dashboard yet spans them. Measure the whole thing, and AI search stops being a black box and starts being a funnel you can actually work.