Google, OpenAI, Microsoft and Anthropic back a shared robots.txt signal against AI training

On 30 October 2026 four major AI companies announced a shared robots.txt signal that lets a website refuse AI training with one entry. Only Anthropic has started so far; the others join later.

Google, OpenAI, Microsoft and Anthropic announced on 30 October 2026 that they back a shared signal in robots.txt that lets a website owner refuse the use of their content for AI training. Until now owners had to look up each crawler by name and write a separate rule for every provider.

The entry uses the user agent "__usage_ai-training" with the rule Disallow: /. Do not delete your existing rules for GPTBot, Google-Extended and others yet: Google states that company-specific controls keep working alongside the new signal, and most providers do not honor it yet. Anthropic has already started, OpenAI plans to implement it in the coming months, Microsoft from 1 February 2027 and Google in 2027.

The signal has limits. It covers training only, so it does not block crawling as such, it cannot retroactively remove data already used for training and it has no legal enforcement. For Google, the Google-Extended token has so far controlled whether already crawled content may be used for training.

For companies this means the choice of letting AI use your content gets simpler soon, but it stays a business decision. Refusing training does not necessarily mean refusing citation in AI search, and AI search brings visitors.