Google Extended: the robots.txt switch for Gemini
Google Extended is a robots.txt token, not a crawler. It tells Google whether content its crawlers already fetch may be used to train future Gemini models and to ground Gemini answers. Google states that it has no separate user agent string.
That is why you will never see it in your access log. The requests still come from Googlebot and Google's other crawlers. The token only changes what Google may do with the pages afterward.
Google-Extended
- Operator
- Type
- AI training crawler
- robots.txt token
Google-Extended- Obeys robots.txt
- Yes, by the operator's documentation. It is a robots.txt control, not a crawler; Google's own crawlers read it
- Official docs
- developers.google.com
What Google Extended controls
Google lists the uses it covers. First, training of future Gemini models, which power the Gemini apps and the Vertex AI API for Gemini. Second, grounding: using your pages as live context for answers in the Gemini apps and for Grounding with Google Search on Vertex AI.
However, just as important is what it leaves alone. Google says the token does not affect whether a site appears in Google Search, and Google Search does not use it as a ranking signal.
How to set the Google Extended rule
Add a group for the token to the robots.txt at your site root. On WordPress, your SEO plugin usually has a robots.txt editor, or you can upload the file by hand.
User-agent: Google-Extended
Disallow: /Also, there is nothing to block at the server level. No request ever carries this name, so an .htaccess or Nginx rule would match nothing. For the same reason, Pagereckon Pro's one-click blocking does not apply here.
Do not block Googlebot by mistake
Still, keep your Googlebot group open. If you disallow Googlebot to keep pages away from Gemini, you also remove them from Google Search, which is a much bigger loss.
What a Google Extended opt-out costs
Per Google, nothing in Search: no change in inclusion and no ranking signal. The trade is inside Gemini. Your pages will not help train future models, and Gemini will not use them to ground its answers.
So a site that wants citations in Gemini answers has a reason to leave the token allowed. A site that cares more about how its content trains models may choose to disallow it. Either way, decide on purpose, since the token also covers grounding and not only training.
How Pagereckon handles Google Extended
Because the token sends no requests, no log-based tool can count Google Extended visits. Pagereckon shows the crawlers that do make requests. Googlebot appears under search crawlers, and the Google watch view shows which pages it crawled and whether it came back after your last edit.
In addition, visits from gemini.google.com count as AI referral traffic, so you can see whether Gemini sends people to you. Open the AI crawler checker to see your crawler mix, or browse the AI bots list for other operators.
Google Extended questions
Does blocking Google Extended hurt my Google rankings?
No. Google says the token does not affect inclusion in Google Search and is not a ranking signal.
Why is Google-Extended not in my server log?
It has no user agent of its own. Google's regular crawlers fetch the pages, and the token only controls how Gemini may use them.
Does the Google Extended token stop AI Overviews?
Google describes it as a control for Gemini models, the Gemini apps and Vertex AI. It does not say it covers features inside Google Search.
Can Pagereckon show Gemini traffic?
Yes. Visits from gemini.google.com appear as AI referral traffic in the free plugin. Download the free plugin to see them.