To opt out of Gemini AI training, add two lines to your robots.txt file:
User-agent: Google-Extended
Disallow: /That's it. That single rule tells Google not to use your site's content for training future Gemini models. It takes effect once Google refreshes its cached copy of your robots.txt — usually within about 24 hours — it costs nothing, and it has no effect on your Google Search rankings. Below is the full picture: what this rule actually controls, what it can't do, how to scope it to parts of your site, and the mistakes that get site owners in trouble.
What Google-Extended actually is (and what it isn't)
Most people assume Google-Extended is a bot. It's not. It's what Google calls a product token — a name that exists only inside robots.txt. No separate crawler called Google-Extended ever visits your site. There are no Google-Extended hits to look for in your server logs.
Instead, regular Googlebot checks this token "in a control capacity." When Googlebot fetches a page, it looks at your robots.txt, sees the Google-Extended group, and uses that as an instruction about how the fetched content may be used downstream. Think of it as a permission label attached to the crawl, not a separate crawler.
This distinction matters for two practical reasons. First, you can't measure Google-Extended with crawl logs — blocking or allowing it changes nothing about who visits your site. Second, you can't block it at the firewall or with .htaccess bot rules, because there is no request with that name to block. Robots.txt is the only mechanism Google offers.
What the token covers, as of September 2026
According to Google's own crawler documentation (updated September 2026), the Google-Extended token governs two uses of your content:
- Training future generations of Gemini models that power Gemini Apps and the Vertex AI API for Gemini — the version of the Gemini API that Google sells through Google Cloud.
- Grounding — pulling content from the Google Search index at prompt time and feeding it to the model to improve the factuality and relevancy of answers in Gemini Apps and in "Grounding with Google Search" on Vertex AI.
That second item is worth pausing on. Grounding is different from training. Training folds your content into the model's weights; grounding retrieves your content at answer time, the way a search snippet is retrieved. Both are governed by the same token, so a single Disallow: / line opts you out of both at once. There is no separate token for "grounding only."
What opting out changes — and what it doesn't
Blocking Google-Extended does not affect your inclusion in Google Search or your rankings. Google says this plainly in its docs, and it's the single most important reassurance for site owners: the token only controls AI uses, not search.
It also does not remove your content from AI Overviews or from Google's AI Mode. This is the most common misunderstanding, so let's be clear: AI Overviews and AI Mode are generated from the standard Googlebot-indexed corpus — the same pages Googlebot crawls for regular search. Google-Extended never governed that content. Blocking it changes nothing there.
If your actual goal is to keep your text out of AI Overviews, you need snippet-control directives like nosnippet, max-snippet, or data-nosnippet. Those work — but they come with a real cost: they also shrink or remove your normal search snippets. That's the trade-off Google currently leaves you with, and for most sites the lost search visibility hurts more than the AI Overview exposure. Weigh that carefully before touching snippet directives.
This matters against a wider backdrop, too. The UK's Competition and Markets Authority designated Google as having "strategic market status" in October 2025, and on March 18, 2026 it opened a consultation that included publishers' AI opt-out rights. Google made an AI opt-out commitment the same day. The political pressure around publisher consent is growing — but the technical mechanism in your hands today is still just this one token.
How to opt out: the exact steps
Here is the full process, start to finish. It takes about ten minutes.
Step 1: Open your current robots.txt
Go to yourdomain.com/robots.txt in a browser. If the file doesn't exist, you'll create one from scratch. If it does, read it carefully before editing — you need to know which rules already exist.
Step 2: Add a dedicated Google-Extended group
Add this block to the file:
User-agent: Google-Extended
Disallow: /Place it as its own group, separate from any User-agent: * block. Google's parser follows only the most specific matching group for a given user agent. That rule has a useful consequence: adding a dedicated Google-Extended group does not disturb your existing wildcard rules, and your wildcard rules can't accidentally override this one.
A complete example with an existing wildcard group looks like this:
User-agent: *
Disallow: /wp-admin/
User-agent: Google-Extended
Disallow: /Step 3: Test before you publish
A typo in robots.txt can hide your entire site from Google, so never upload blind. Paste your updated file into a robots.txt tester — CrawlGuards' free robots tester lets you check exactly how Google will read each line — and confirm that Googlebot's access is untouched while Google-Extended is blocked.
Step 4: Upload and wait
Save the file at the root of your site (yourdomain.com/robots.txt — subdirectories don't count). Google generally caches robots.txt for up to about 24 hours, so your change applies on the next refresh, not instantly. If you're nervous, re-check after a day. You can also build the rule visually instead of hand-writing it: CrawlGuards' free robots.txt generator produces the correct group for Google-Extended without syntax guesswork.
Partial opt-outs: blocking only parts of your site
You don't have to go all-in. The Disallow value can target specific paths, which means you can opt some sections of your site out of Gemini training while leaving others available.
A common setup for a news site: block today's reporting but leave the archive open. That looks like this:
User-agent: Google-Extended
Disallow: /news/
Disallow: /2026/Or a creator who wants their free articles excluded but keeps documentation available for grounding:
User-agent: Google-Extended
Disallow: /blog/
Allow: /docs/Note the Allow line: it only matters when it refines a broader Disallow. In the example above, Allow: /docs/ has no practical effect because /docs/ was never disallowed — it's shown to illustrate the syntax, since many real-world files combine broad blocks with narrow exceptions. When you do mix them, remember the specificity rule: Google's parser resolves conflicts by picking the longest matching path, so Allow: /docs/guide.pdf beats Disallow: /docs/.
How long does it take to work?
Plan on roughly 24 hours. Robots.txt is cached by Google, and the new rule applies when Google fetches a fresh copy — not the moment you save the file. Don't panic if a tester shows the new rule while Google's behavior seems unchanged; the cache is the reason.
There is also a second, less obvious time dimension: opting out affects future use of your content. If your pages were already used to train an existing Gemini model, no robots.txt change can make the model unlearn them. The token is a forward-looking control. The sooner you add it, the less of your future content flows into training pipelines.
Common mistakes to avoid
1. Accidentally blocking Googlebot
This is the dangerous one. A rule like this:
User-agent: Googlebot
Disallow: /removes your whole site from Google Search. It looks almost identical to the Google-Extended rule, which is exactly why copy-paste errors happen. Triple-check the user-agent line. After uploading, test your file and confirm Googlebot still has access. One misplaced token name is all it takes to vanish from search.
2. Thinking Google-Extended is a separate crawler
Because it's not a crawler, none of the crawler-blocking tricks apply. Blocking the name in a firewall, a security plugin, or .htaccess does nothing — there is no request to intercept. Robots.txt is the mechanism, full stop.
3. Catch-all wildcard side effects
A wildcard group like this:
User-agent: *
Disallow: /private/applies to every crawler without a more specific group. The good news: once you add a dedicated Google-Extended group, that group takes precedence for the token and your wildcard rule no longer touches it. But the reverse bites people too — if you put Disallow: / under User-agent: * thinking it only affects AI bots, you've just blocked every crawler on the web from your entire site. Wildcards are blunt instruments. Name your targets.
4. Expecting instant or retroactive results
As covered above: ~24 hours for the rule to be picked up, and no effect on models already trained. Robots.txt is a "going forward" signal, not a time machine.
Should you opt out? Pros and cons by site type
Blocking Google-Extended is free and doesn't hurt search rankings, so the decision comes down to how you feel about your content feeding Gemini's training and grounding pipelines. Different sites land in different places.
News publishers and original reporting. Strongest case for opting out. Your content is expensive to produce, it's the exact high-quality material that improves models, and the value exchange is murky. Many publishers also resent grounding, where their facts appear in AI answers without a visit to their site. Opting out of training is the minimum; the grounding question is a business call.
Independent creators and bloggers. Moderate case. If your work is distinctive — original analysis, creative writing, photography — training use without credit or compensation is a legitimate concern. The counter-argument is thin: blocking costs you nothing in search, so there's little downside to flipping the switch.
E-commerce and product sites. Weak case for, weak case against. Product descriptions have limited training value, and grounding can actually surface your products in Gemini answers with citations. Most shops can safely skip this decision or leave the token allowed.
Local businesses. Minimal relevance. Your hours, address, and service pages aren't valuable training data, and grounding helps AI assistants answer "where can I get X near me" accurately. Opting out buys you almost nothing.
Documentation and reference sites. A genuine split. Grounding makes your docs more discoverable through Gemini with citations that send traffic your way; training use gives you nothing back. Some sites keep grounding benefits by staying opted in; others block everything on principle. This is the one category where I'd say the answer depends on your monetization model, not a general rule.
One more consideration: the gesture has collective weight. Individual opt-outs are about your content; widespread opt-outs shape what AI companies can train on next. Whether that motivates you is personal, but it's real.
FAQ
Will blocking Google-Extended hurt my Google rankings?
No. Google states that the Google-Extended token has no effect on search inclusion or ranking. It only controls the two AI uses described above: Gemini model training and grounding. Your search traffic is untouched.
Does it remove my content from AI Overviews?
No. AI Overviews and AI Mode are built from the regular Googlebot-indexed corpus, which Google-Extended never governed. Limiting AI Overviews requires snippet-control directives like nosnippet — which also reduce your normal search snippets, so think hard before using them.
Is Google-Extended a bot that crawls my website?
No. It's a product token, not a crawler. Regular Googlebot checks it "in a control capacity" when reading your robots.txt. You won't find Google-Extended in your server logs, and firewall or .htaccess blocks can't affect it. Robots.txt is the only control.
How do I know the block is working?
First, confirm your robots.txt is live at yourdomain.com/robots.txt and contains the two-line group. Second, run it through a tester — this free robots tester shows exactly how Google parses each line. Beyond that, there is no dashboard or confirmation from Google; the signal is trusted on the honor system, like all robots.txt rules.
Can I block Gemini training but keep ChatGPT training allowed?
Yes — they're separate tokens. OpenAI uses GPTBot for training, a completely different user-agent name with its own robots.txt group. Blocking one has no effect on the other, so you can make independent choices for each AI company.
Should I just use "User-agent: *" with "Disallow: /" to block all AI?
Only if you mean it — that combination blocks every crawler from your entire site, including Googlebot and Bingbot, and your search traffic will collapse. For AI-specific opt-outs, name each token explicitly: Google-Extended for Google, GPTBot for OpenAI, and so on. Wildcards are the bluntest tool in the file; use them with care.
For background on the token's history and ongoing debate, see this Google-Extended explainer, and for the authoritative reference, Google's official crawler documentation. If you run a site with AI-crawler concerns, the CrawlGuards blog covers new crawler tokens and opt-out mechanics as they change.
