Google plans to honor a shared robots.txt token for opting out of AI training

Google says it will read a shared __usage_ai-training robots.txt token as a Google-Extended opt-out, aiming for 2027 alongside other AI providers.

Google said on Oct 30, 2026 that it plans to support a shared robots.txt policy token that tells participating AI providers not to train on your content. That would replace one rule per vendor with one group. If you run a site, docs portal or client blog, you can add it now, but nobody has said they honor it yet.

The token comes from an individual IETF Internet-Draft by Anthropic's Kevin Kelley. The Datatracker notes the draft is not endorsed by the IETF and has no formal standing. It adds no new grammar. You name the token on the User-Agent line:

# Opt out of AI training except the press kit (example from the draft)
User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/

There's a catch. Under the draft, a User-Agent: * group never matches the token, so a blanket block says nothing about training (Search Engine Journal, Nov 1, 2026).

Google says that if you use the token and don't name Google-Extended, it will treat that as a Google-Extended disallow. Existing vendor controls keep working. Its post aims for support "in 2027" with other providers.

Other providers are on different timelines, according to PPC Land (Oct 31, 2026):

In practice, keep your GPTBot and Google-Extended rules alongside the new group until each vendor confirms it reads the token, as H2S Media advises. It is a preference, not an access control. Whether it covers retrieval at answer time is unclear.

Sources