Splitting AI Search Measurement Into Five Layers

Tomohiro Iida · Published: June 30, 2026Last updated: September 30, 2026

Appearing inside a generative AI feature within Google Search, being cited by an external AI service, receiving traffic through AI, branded search, and an actual sales conversation are separate events. Rather than collapsing them into one number, this article sets out which data answers which question. It is written for web and IT teams who want to measure how a company appears in AI search.

Layer 1: Google generative-AI impressions and overall Search performance

Google Search Console exposes this layer in two related views. The dedicated generative-AI performance report isolates impressions in AI Overviews and AI Mode, while the main Search performance report continues to include those appearances within the site’s overall Search performance data.

On June 3, 2026, Google announced a dedicated performance report for its generative AI features, and the help page states it rolled out worldwide on August 31, 2026. What this report shows is impressions for AI Overviews and AI Mode, broken down by page, country, date, and device. A query dimension is not documented, so it does not tell you which questions you appeared for, and clicks, CTR, and position are not documented as metrics of this dedicated report either — those remain in the main Search performance report, which still includes this data. The help page also retains caveats about rollout and minimum impressions, so check your own property in the console rather than assuming it is visible. Record the date you checked, since screens and availability can change.

Google Search Central: AI features and your websiteChecked September 30, 2026.

Search Console Help: Generative AI performance reportChecked September 30, 2026.

Metrics to check: in the dedicated report, generative-AI impressions and their page, country, date, and device dimensions; in the main Search performance report, the overall clicks, CTR, position, pages, and period used for comparison. Do not attribute those overall click/CTR/position figures to the dedicated AI report.

What this layer cannot tell you: which statement in the answer your page was used for, whether an external AI service cited you, whether a user who did not click went on to make an inquiry, and whether an increase in appearances is the result of a particular change.

A guide to AI OverviewsHow the feature works and how to prepare for it.

Layer 2: citations and mentions in external AI services

This layer observes whether your pages appeared as a source in ChatGPT search, Perplexity, and similar services. No off-the-shelf report exists here, so you end up running the questions and storing the results yourself. Answers are non-deterministic and can differ for the same question, so no conclusion is drawn from a single observation.

Split what you store into the conditions of the observation and what was observed. If the conditions are missing, you cannot later confirm whether two numbers came from comparable observations.

Condition: question text and question-set version

Bump the version whenever questions are added or changed, and record which version produced each result.

Condition: branded or non-branded, and stage

Problem awareness, comparison, purchase, and so on.

Condition: language (locale)

Keep the region part as well, for example ja-JP.

Condition: requested country or region

The value you specified as an input to the observation.

Condition: confirmed country or region

Record it only when the answer, its sources, or the provider response confirm it; otherwise record it as unknown.

Condition: AI service, model, and search mode

Including whether web search was used.

Condition: run timestamp

When the observation was made.

Result: observation status

Observed, unsupported language or region, failed run, or result could not be determined. Keep the error detail when a run fails.

Result: cited URLs

The source URLs attached to the answer.

Result: mention

Whether the company or service name appeared in the answer body.

Result: reflection in the answer body

Whether content from your pages was used in the answer.

Result: inaccuracy

Whether company or service details were described wrongly.

Keep "cited" and "reflected in the answer" apart. Appearing in a list of reference links and having your distinguishing points used in the body of the answer mean different things.

Citation and absorption into the answer are different things

Put the requested region and the confirmed region in separate fields. Specifying a region is only an input to the observation; whether the AI service actually used search results or sources from that region usually cannot be verified from outside. If it cannot be confirmed, leave it as unknown and do not copy the requested value into the confirmed field.

Record "no citation" only when the observation succeeded, the answer could be judged, and your URL was not among the sources. Observations where the AI service does not support the target language or region, where the run failed, or where the result could not be determined are not recorded as zero citations; keep them as counts by status. Treating "not found" and "not measured" as the same zero makes it impossible to tell whether citations fell or measurement stopped.

Comparisons over time are valid only between observations with the same question-set version, language, requested region, confirmed region, AI service, model, and search mode. If any of these changes, treat the result as a separate series. Observations with the same requested region but different confirmed regions belong to different series. An unknown confirmed region is its own state: compare unknown-region observations only with other unknown-region observations under otherwise identical conditions, and never mix them silently into a series with a confirmed region. An unknown confirmed region does not invalidate an observation whose result could be judged, and it does not change the denominator: only unsupported, failed, and undetermined observations are excluded from it. The reasoning behind observation identity, the number of repetitions, confidence intervals, and how to read a score are covered in the article on measuring AI visibility.

Measuring AI visibility: repetitions, confidence intervals, and the limits of a scoreObservation identity, repetitions, and confidence intervals.

Layer 3: site traffic from AI services

This layer covers only users who actually reached the site. In GA4 or a similar tool, you check visits referred by ChatGPT, Perplexity, Gemini, and so on.

The channel definitions come with a caveat. GA4's default channel group places traffic from AI assistants such as ChatGPT in a separate AI Assistant channel. Google AI Overviews and AI Mode are excluded from that channel, so they do not appear there. Our own dashboard does not rely on channel names: it aggregates on an exact-match list of referral sources (GA4 sessionSource). New AI services we cannot identify fall outside the aggregation by design, erring towards not inflating AI traffic.

Google Analytics Help: Default channel groupThe AI Assistant channel covers referral sources such as ChatGPT, Gemini, Deepseek, Copilot, or Grok and excludes Google AI Overviews and AI Mode. Checked September 30, 2026.

On top of that, in-app browsers, redirects, and privacy controls can strip the referrer. On a finalized daily basis it is observed hours to one or two days late, which makes it a lagging indicator.

Metrics to check: sessions, landing pages, dwell time and scroll, arrival at the main CTA, inquiry starts, inquiry completions, and new versus returning.

Even when AI referral traffic is small, some users learn a company name from an AI answer and search for it by name later. Do not judge the value from traffic alone.

How much traffic actually comes from AI search?Measured figures from our own GA4 property.

Layer 4: branded search and comparison behaviour

Users who learn a company or service name from an AI answer often go back to a search engine to check it. In Search Console, track branded queries such as the company name, service names, and the name of the representative on a fixed basis. Asking on the inquiry form, as an optional field, where someone heard about you is another route.

That said, an increase in branded search cannot be attributed to AI search alone. Events, social media, referrals, advertising, and sales activity are all moving at the same time.

How to grow branded search

Layer 5: inquiries, sales conversations, and orders

This layer confirms whether anything reached the business. What gets looked at: inquiry counts, qualified opportunities, what was asked about, orders, contract value, the first touchpoint the person reported, the pages they read, and the number of days to the inquiry.

When implementing measurement, decide the form event names first. On netsujo.jp we record the start of input as form_start, a submission attempt as form_submit, a submission the server accepted successfully as generate_lead, and a failed submission as form_submit_error. The form_submit that GA4 enhanced measurement picks up automatically is not reliable, so the denominator for a success rate uses events we fire ourselves.

At this stage the set includes cases that cannot be attributed to AI search with any certainty. Where UTM parameters or a referrer survive, record them and reconcile self-reported answers against access logs.

Connecting SEO and AI search work to inquiries: how to measure it

Designing the citation metrics and the query set

From here on, the subject is how to implement Layer 2, the layer with no off-the-shelf report. Two base metrics carry it.

Citation presence

Did the domain appear at least once as a source or an in-text link in the answer to a given query? A binary, true or false.

Citation share

The proportion of a target query set in which the company was cited. For example, cited in 8 of 25 target queries gives a share of 32 percent. Queries that could not be judged because the market was unsupported or the run failed are not counted as uncited; report their count separately.

Citation depth (supplementary)

The average position at which the company appeared as a source within the answer. It rests on the hypothesis that appearing higher carries more weight.

Competitor delta (supplementary)

The gap between your share and a main competitor's share on the same query set, used to tell whether you rose or the comparison fell.

These depend heavily on the particular query set. What carries meaning is tracking change over time under the same observation conditions (query-set version, language, requested and confirmed region, engine, model, and search mode), rather than the absolute value. Accuracy is largely governed by how the query set is designed; model variance, region, login state, and API changes matter too. For each topic cluster, fix 10 to 30 representative queries that a real user might type. On netsujo.jp the line-up looks roughly like this.

• Branded and proximity queries: "Web3 development company Kyoto", "system development Kyoto"
• Term-definition queries: "what is DID", "what is RWA"
• Problem-solving queries: "causes of PoC failure", "smart contract audit"

• Keep the set fixed. Adding and removing queries week to week makes it impossible to tell whether a move in share came from content improvement or from changing the set. When you do change it, bump the version and do not join it to the old series.
• Build in spelling variation for Japanese. Kanji, kana, and romanized forms can change the answer, so include variants within reason.
• Do not load up on branded queries. Queries containing the company name are cited easily and make the share look higher than it is. Keep an eye on the proportion of general problem-oriented terms.

Putting the cluster definitions in a versioned configuration file that the code reads means the same set can be run when you add another engine.

Judging citations and tracking them with snapshots

The judgment is most straightforward on engines that return an array of source URLs in the response. Taking the Perplexity Sonar API as an example, the minimal logic sends the question as messages, receives the answer plus citations, then parses each cited URL and matches the hostname either exactly or as a subdomain of the brand host.

Two things need care: the source array can come back null or empty, and a naive substring match produces false positives, matching a spoofed host such as netsujo.jp.evil.com. Parse the URL and judge on the hostname. It is more stable to split judgment in two: a source-list check, which looks for the domain inside the structured citation array, and a body-text check, which looks for mentions of the brand name or domain in the answer text. The former has fewer false positives; the latter catches cases where the name appears without a source link.

An empty source array can still be judged as "no citation" when the response came back normally. When no response was obtained because of an error or timeout, or when you cannot confirm that search actually ran, separate that out as an observation status before judging. Only responses from successful observations go into the judging function.

Engines differ in how they expose sources. For ChatGPT the wiring depends on what you are measuring: browser automation with a login for answers in the UI, the official API citation annotations for web-search responses via the API. Gemini returns source annotations through Grounding with Google Search, so structured retrieval via API is possible. AI Overviews has no official Google API, leaving an approximation through third-party SERP retrieval services, with reproducibility and each vendor's terms of service to watch.

The observation we publish runs on a single line, using OpenAI web search. An adapter for Perplexity Sonar is implemented but disabled in the published observation. We have not reached fully automated measurement across engines, and we operate on the assumption of widening it in stages.

Each run is stored as a per-query snapshot. The key point is retaining only the extracted citation information and the observation conditions rather than the full response, both for storage volume and for privacy. One query's record holds roughly the run timestamp, the query-set version, the query, the locale, the requested region, the confirmed region, the engine, the model, the search mode, the observation status and any error, the brand hosts, the cited URLs, whether the brand was cited, whether it was mentioned, whether content was reflected in the answer, any inaccuracy, and the citation count.

A confirmed region left empty means "could not be confirmed"; it is never filled in from the requested region. The status is observed, or one of unsupported (the target language or region is not supported), failed (the run failed), and undetermined (the result could not be judged). Reflection in the answer and inaccuracy are filled in after those checks are done.

These accumulate by date, engine, and query, and each week the latest is compared with the previous snapshot to produce the week-on-week change in citation share, queries newly cited, queries that lost their citation, and the change in the gap against comparisons. Only snapshots with the same question-set version, language, requested region, confirmed region (unknown only with unknown), engine, model, and search mode are compared. Rows whose status is anything other than observed are not counted as queries that lost their citation; they are listed as counts by status. Run intervals are tuned to each engine's rate limits.

Looping uncited queries back into content

The value of measurement is not in looking at numbers; it is in turning uncited queries into an input for improvement. When a query loses its citation, or has never been cited at all, review the corresponding page on three points.

Passage structure

Is the conclusion or definition gathered at the top of the paragraph? Generative engines tend to lift an easily quotable passage, so points buried deep in the text may be less likely to be picked up.

Machine readability

Are definitions and FAQs machine-readable in a form that matches the visible text?

Crawlability

Can AI crawlers reach the page in the first place?

Deciding whether to allow or block AI crawlersHow to design the permissions.

Seven reasons AI search is not citing your pages

The basics of generative engine optimization (GEO)How to structure clear, first-party content without assuming a citation effect.

Do not collapse this into one "AI visibility score"

Even when an index is used as a summary for management, the underlying metrics have to stay. The layers do not necessarily move in the same direction.

Citation rate in external AI can rise without site traffic rising. Appearances inside Google can rise without producing sales conversations if the inquiry page is hard to follow. Conversely, a company may not be cited by AI at all while branded search and referrals from existing customers grow. A composite figure is an aid for seeing the situation at a glance. Improvement decisions are made on the layer-level metrics.

Measuring AI visibility: repetitions, confidence intervals, and the limits of a scoreGoes deeper into aligning observation conditions, repetitions, confidence intervals, and how to read a score.

Pulling it together in a monthly table

To see everything on one page while keeping the layers apart, the monthly summary takes the following shape. Because there is a lag between a change and a result, always record the publication date.

State what cannot be measured, up front

Nothing about AI search measurement gives a complete picture. To keep operating decisions from going wrong, write down what cannot be measured first.

• Users who read the AI answer and stopped there
• Visits whose referrer was lost
• References within the same answer that were not cited
• Candidate generation inside the AI service
• The region the AI service actually used, when the answer and response do not confirm it
• The reason an answer changed
• Exact attribution for an inquiry that passed through several touchpoints

Observation has limits of its own. Answers vary run to run for the same query, so a single false does not establish that you are not cited. Engines that return a source array and engines where retrieving sources is itself an approximation do not measure to the same accuracy, so simply summing share across engines makes the reading unstable. The same applies to summing observations from different languages or regions. With a small query set, share swings on a few queries: with 30 queries, one change moves it by more than 3 points. How each vendor returns sources, and their parameters, change on their own schedule, so the judging logic has to be kept up to date.

For that reason we avoid asserting causation of the form "this change raised the citation rate by N points". What the measurement can show is relative movement under fixed conditions, and where the queries needing improvement are, and no more. Making the observed range explicit is more useful for decisions than hiding the unmeasurable part behind a precise-looking score.

How Netsujo SIGNAL approaches measurement

Netsujo SIGNAL treats appearances inside Google, citations in external AI, AI referral traffic, branded search, and on-site behaviour separately. The free diagnosis shows what can be observed from public information; ongoing improvement connects Search Console, GA4, and the site change history to identify which stage the problem sits at.

For Layer 2, what we implemented is a design that does not estimate. Rather than assuming we are probably being cited, we actually call a search-enabled AI and parse the returned answer and citation URLs as they come. What our IT engineers built by hand is a three-stage flow that goes straight for this primary-source data.

• Calling a search-enabled AI for real: we send an AI with search (in our implementation, OpenAI web search) the kind of natural-language question a user would actually ask, and receive the answer text together with the source URLs it drew on.
• Parsing the citation URLs: we extract URLs from the citation annotations attached to the answer and normalize the host names. Judging by host rather than substring is the same defence against spoofed hosts described earlier.
• Counting citations of your own domain: we count citations whose normalized host matches the target domain. That is the number of times the company was actually cited as a source within the answer. References alone, and appearances in search results, are not included in that count.

The shape of the questions used for observation is part of the same design. Rather than a string of SEO keywords, they are conversational sentences such as "What kind of company is X?" or "Which company should I consult to solve X?" Generative engines are used conversationally, so observing with inputs close to real use is what keeps the measurement meaningful.

SIGNAL Lab methodologyThe models used and the run conditions are published here.

The line the implementation drew is the boundary of what may be disclosed. SIGNAL's free diagnosis returns the count of times your own domain was cited in the answers, together with the questions and conditions used for the observation. The count is primary-source data for grasping where you stand. Reading the breakdown of citing sources and turning it into an order of improvements sits within the ongoing-improvement scope. The count alone is enough of a starting point to see where a company stands.

The other thing the implementation separated is "observed, but zero citations" from "could not observe at all". Presenting a run where observation failed entirely, because of a provider-side outage for instance, as zero citations with the diagnosis completed normally would be a fake zero dressed up as measurement. Whether an observation succeeded is held as state, and a run that did not succeed is never displayed as one that did. Each question is also run more than once, the result is held as a rate (the number of times cited divided by the number of successful observations), and the denominator, how many observations succeeded, accompanies the result. How to decide the number of repetitions, and the confidence interval that goes with the rate, are covered in the article on measuring AI visibility.

Measuring AI visibility: repetitions, confidence intervals, and the limits of a score

The goal is not to raise an AI visibility score. It is to reach a state where the people who need your company can find it, understand it, compare it, and get in touch.

Our SEO and AI search practice logThe overall approach taken on netsujo.jp.

Frequently asked questions

Can I isolate the AI Overviews portion in Search Console?

Some of it can. Google announced a dedicated generative-AI performance report on June 3, 2026, and the help page states a worldwide rollout on August 31, 2026. It covers impressions for AI Overviews and AI Mode with page, country, date, and device dimensions. A query dimension is not documented, and clicks, CTR, and position are not documented as metrics of this dedicated report; those stay in the main Search performance report, which continues to include this data. Rollout and minimum-impression caveats remain in the help page, so confirm in your own console whether it is available to your property (sources: developers.google.com/search/blog/2026/06/gen-ai-performance-reports, support.google.com/webmasters/answer/16984139, checked September 30, 2026).

Can every engine be measured automatically?

The observation we publish runs on a single line, using OpenAI web search. An adapter for Perplexity Sonar is implemented but disabled in the published observation. Answers in the ChatGPT UI, Gemini, and AI Overviews each expose sources differently, and we have not reached fully automated measurement across engines. The models used and the run conditions are published on the methodology page.

If I specify a region when observing, can I call it the result for that region?

Not on that basis alone. Specifying a region is an input to the observation, and whether the AI service actually used search results or sources from that region usually cannot be verified from outside. Keep the requested region and the confirmed region as separate fields in the observation log, and record the confirmed region as unknown if it cannot be confirmed. Observations where the target language or region was unsupported, or where the run failed, are not counted as zero citations; they are recorded as a status.

How do citation presence and citation share differ?

Presence is the binary question of whether you were cited for one query; share is the proportion of the whole query set in which you were cited. It helps to think of share as presence aggregated into a proportion.

How often should measurement run?

Weekly is an easy starting value to operate, as our own guideline. It is not a universal right answer. Because generative engine answers vary, important queries are run several times rather than judged on one result. Frequency also gets adjusted for the number of queries and API cost. Around a large content change, taking a baseline and re-measuring two weeks later suits the purpose.

Can I roll this into a single AI visibility score for management reporting?

Using a combined index as an aid is fine, but keep the underlying layer-level metrics. Citations rising while traffic does not move, or traffic rising while sales conversations do not, genuinely happens. Improvement decisions are made on the layer-level metrics.