Do Google AI Overviews Reduce Website Clicks? What the Experiment Actually Proves

Yes. A preregistered field experiment found that forcing Google searches into AI Mode caused fewer external clicks. The estimated reduction was 18.8 percentage points versus current Search. But the widely repeated AI Overview figure needs a technical correction. The 8.8-point result was a compliance-adjusted estimate. Assignment to the No AI condition produced a 2.7-point effect that was not statistically significant.

These findings are important causal evidence. They are not a forecast that every publisher will lose 18.8% of traffic.

Key takeaways

• The study observed 1,100 US participants using Google in ordinary browsing after random assignment.
• Forced AI Mode reduced external click-through rate by 18.8 percentage points, with a 95% confidence interval from 22.2 points lower to 15.3 points lower.
• The No AI assignment increased click-through by 2.7 points, but its interval included zero. The stronger 8.8-point number is a local average treatment effect for successful AI Overview removal.
• Google changed its AI Overview HTML during the experiment. The blocker worked for 51.1% of AI Overview exposures overall.
• User-side removal, publisher opt-out and normal product adoption are three different interventions. The experiment directly tested only the first and a forced version of the third.

How was the Google AI search experiment designed?

The August 18, 2026 arXiv preprint reports a randomized field experiment. Researchers recruited US adults who used Chrome and Google Search as their primary browser and search engine. Everyone first completed a three-day current-Search baseline. Participants were then assigned to one of three seven-day conditions:

1. No AI Search: a browser extension attempted to hide AI Overviews and redirected AI Mode searches to standard Search.
2. Current Search: Google operated normally, so AI Overviews could appear and AI Mode remained available.
3. AI Mode Search: nearly every Google search was redirected into AI Mode.

The behavioural analysis included 1,100 participants who made at least one search during treatment. The final survey included 956 people. The sample was useful but not nationally representative: 75% were under 45 and 87% had at least some college education.

What exactly did the experiment count as a click-through?

The outcome was not a conventional Search Console CTR. The researchers calculated external clicks from Google divided by Google searches for each user. They excluded clicks to google.com and searches on vertical pages such as Images, News and Shopping.

Each turn inside an AI Mode conversation counted as a search in the main specification. That choice can enlarge the denominator when a conversation has several turns. The authors tested an alternative definition where one conversation counted as one search; forced AI Mode still reduced click-through by 14.9 percentage points. The direction and statistical significance survived that robustness check.

This matters because “CTR” can describe different denominators. A user-level experimental outcome, a Search Console property ratio and a page-level organic CTR are not interchangeable.

What did the click estimates actually show?

The AI Mode arm also produced 0.92 fewer search sessions per day and 0.43 more minutes per session. Participants spent longer inside each session while starting fewer sessions and leaving Google less often. Click incidence fell for news sites, Reddit and Wikipedia. Ad clicks also fell. However, AI Mode did not show ads during the March 2026 experiment. That result does not describe a later commercial design.

Why is the 8.8-point AI Overview result easy to misread?

Intent to treat (ITT) compares people according to their random assignment, whether or not the intervention worked every time. Randomization makes ITT the cleanest estimate of offering or assigning a treatment.

Local average treatment effect (LATE) estimates the effect among people whose exposure changed because of assignment. It requires additional assumptions about compliance.

The distinction became decisive when Google changed the HTML structure of AI Overviews during the study. The extension hid about 90% of detected AI Overviews on day one, then its success fell to zero. Across the experiment, only 51.1% of AI Overview exposures were successfully hidden. The resulting No AI ITT estimate was +2.7 points and statistically inconclusive; the LATE estimate was +8.8 points and statistically significant.

The correct headline is not a claim that AI Overviews cut every site's traffic by 8.8%. Successful removal increased external clicking for the compliant population estimated by the study's model.

Does the experiment contradict Google's traffic claims?

Not directly. Google said in August 2025 that total organic click volume from Search was “relatively stable year-over-year” and that average click quality increased. Google defined a quality click as one where the user did not quickly return to Search. That is an aggregate platform observation with an undisclosed dataset, not a randomized estimate of AI exposure.

The experiment asks what happened when comparable users received different interfaces. Google's statement asks what happened to total traffic after query volume, user behaviour, ranking distribution and the mix of websites all changed together. Both claims can be true if AI suppresses clicks on exposed searches while new queries or redistribution offset some aggregate loss.

Neither source establishes the effect on one publisher. Google's “quality click” definition also does not prove a subscription, lead or sale.

Does the study prove that publishers should opt out?

No. Since August 31, 2026, Google says its generative AI reports and inclusion control are available to websites worldwide. An opted-out site receives no impressions or traffic from AI Overviews, AI Mode or generative features in Discover. Google says the setting is not a ranking signal outside those features.

That publisher control is not the experiment's No AI condition. The experiment removed AI features from a user's interface. A publisher opt-out removes one source while the generative answer can remain and cite alternatives. The excluded publisher could lose AI exposure without recovering a classic-search click.

An opt-out test therefore measures a distribution decision, not the causal effect reported by the experiment.

What is Xtrusio's model for interpreting possible click loss?

The useful model is an exposure funnel, not one market-wide multiplier.

Observed business impact = affected-query exposure × change in external click yield × value per resulting visit.

Each term must come from the publisher's own comparable cohort. The experiment informs the direction and plausible importance of the middle term. It does not supply a site's exposure rate, query mix or visitor value.

The Xtrusio Google AI Overview tracking workflow explains how to preserve these layers without merging them. The GEO campaign measurement guide supplies the wider metric dictionary.

How should a publisher test its own exposure?

Start with a four-week baseline and ten to thirty commercially important landing pages.

1. Freeze major content and template changes during the comparison window where practical.
2. Group pages by search intent: answer-complete, research-heavy and action-dependent.
3. Export complete Search Console dates and preserve the reporting timezone.
4. Record generative impressions separately from total Search impressions and clicks.
5. Re-run a fixed buyer-question cohort with the same location and device settings.
6. Compare qualified outcomes per 1,000 Search impressions, not clicks alone.
7. Use similar pages without a material exposure change as a matched reference group.

That matched comparison is a diagnostic, not automatically a causal experiment. Exposure is not randomly assigned, and Google may change ranking, layout or query eligibility at the same time. Label the result accordingly.

The practical response is not to publish more commodity summaries. Build pages whose value survives the summary: original data, primary documents, executable tools, decision logic, expert experience and next-step workflows. Those assets can earn both citation visibility and a reason to click.

Sources reviewed

1. Wang et al., AI in Search Reduces Publisher Referrals Without Improving User Experience
2. Google, AI in Search is driving more queries and higher quality clicks
3. Google Search Central, Generative AI performance reports in Search Console
4. Google, New opportunities, control and insights for website owners
5. Google Search Central, AI features and your website