Google, Bing and Anthropic Back One AI Training Opt-Out for Robots.txt

What the Shared AI Training Signal Is

The shared AI training signal is a robots.txt group with a reserved name, __usage_ai-training, that any AI company can agree to read. A site that writes "User-Agent: __usage_ai-training" followed by "Disallow: /" is saying no AI training on the whole site. The draft standard, "Machine-Readable Training Preference Signal", was written by Kevin Kelley of Anthropic and filed at the IETF, the body that publishes internet standards, on October 30.

Path rules work the way they always have. The draft's own example blocks everything except a press kit folder. One detail will catch people out: the draft says a "User-Agent: *" group must not be read as matching the new token. A site has to name __usage_ai-training on its own line.

What Google Said About the AI Training Opt-Out

Google said the AI training opt-out will let websites set their preferences "across multiple AI providers in a single step," in a Search Central post by John Mueller. When a site opts out with the shared token and does not name Google-Extended, Google "plans to interpret that as if the publisher had disallowed Google-Extended." A rule written for Google-Extended still wins.

It is not live at Google yet. Google's crawler documentation says it "intends to support" the signal "in early 2027", and the blog post says 2027. Headlines saying Google has adopted it went further than Google did.

Bing and Anthropic Dates

Bing and Anthropic dates differ from Google's. Microsoft Bing will honor the new preferences as it crawls from February 1, 2027, and splits two choices: AI training, set in robots.txt or on the page, and AI grounding, set on the page only. After March 1, 2027, the old NoArchive and NoCache tags stop counting as AI preferences in Bing. Bing's own line on what a site gets back: "Neither choice guarantees inclusion, traffic, or payment."

Anthropic's help page says Claude already recognizes the signal as of October 30, and a robots.txt rule that names ClaudeBot overrides the general one. OpenAI also backs the signal, according to news coverage of its announcement, but its timeline is less specific.

Does It Touch Google Search Rankings?

Google Search rankings are not touched by it, going by Google's own documentation. Google-Extended, which the shared opt-out maps onto at Google, "does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search." So blocking AI training is not blocking Googlebot. Google's post does say that sites which allow Google-Extended can appear in Gemini Apps answers through grounding and get clicks from them.

What to Do Now

What to do now is mostly to decide. If you already block GPTBot, ClaudeBot and Google-Extended one by one, those rules keep working, and the shared group can sit beside them. If you want your pages cited in AI answers, read Bing's training and grounding split before blocking anything, because they are separate switches there. Sites still using NoArchive or NoCache as an AI opt-out have until March 1, 2027 to move to the new tags for Bing.

For the wider fight over AI and links, the GEO and AI search shelf collects the channels and podcasts that cover it every week.