One robots.txt rule to opt out of AI training: what it covers

On October 30, 2026, four of the biggest AI companies, Google, OpenAI, Microsoft and Anthropic, said they will honor one shared robots.txt signal for opting out of generative AI training. Until now a website owner who did not want their pages used for training had to know a separate crawler name for each company and keep the list up to date. The new approach replaces that with a single rule.

It is a real step, but a smaller one than some headlines suggest. Here is what is known, what is still open, and what it means whether you publish a site or just write a lot of text for a living.

What was announced

The shared signal is a robots.txt group with a special name, __usage_ai-training. A site that wants out adds two lines:

`` User-Agent: __usage_ai-training Disallow: / ``

You can also limit the rule to certain folders instead of the whole site. The syntax is ordinary robots.txt, so existing tools and habits keep working. Coverage describes the idea as an individual proposal at the IETF (the body that writes internet standards), written by an Anthropic engineer. It is a draft, not a finished standard.

Who does what, and when

This is the part to read carefully, because the reports do not fully agree and the companies are moving at different speeds.

• Anthropic is reported to have started supporting the token already.
• OpenAI is reported to plan support in the coming months.
• Microsoft is reported to start on February 1, 2027.
• Google is aiming for 2027, with some reports saying early 2027.

Treat these dates as approximate. We could not open every company's own documentation from our research session, so check the official crawler pages of each company before you rely on a date. Until a company confirms support is live, keep your existing rules for its crawlers (for example GPTBot, Google-Extended and Bingbot controls) in place. The new token is an addition, not a replacement.

What it does not do

• It is not retroactive. It tells crawlers what to do from now on. It does not remove pages that were already collected into datasets or models.
• It is about training, not search. Whether a search engine or an AI answer can quote and link your page is a separate question with separate controls. Reports differ on how Google and Microsoft treat AI answers (grounding) under the new token, so read the vendors' documentation before assuming.
• It is a request, not a lock. robots.txt has always worked on trust. Companies that honor it will; others may not. Some commentators call the move largely symbolic for that reason, given earlier reports of crawlers ignoring robots.txt.
• It is not law. It does not replace copyright rules or the EU's and other regulators' requirements. If you want to see how regulation is moving, our piece on the US executive order on vetting frontier models covers the other side of the picture.

Should you add it?

If you run a blog, a portfolio or a company site and you do not want the text used to train models, adding two lines costs nothing and does no harm to ordinary search indexing. Just do not remove your existing per-company rules yet. If you want your content to appear in AI answers, think before you block, because some vendors may treat training and answer features together.

If you are a writer, lawyer, consultant or founder, the lesson goes beyond robots.txt: the default for text on the open web is that someone may read it. The default for your private drafts, calls and notes should be the opposite.

What this means for your own words

The robots.txt story is about public pages. Most of what professionals produce is not public: drafts, client emails, call notes, voice memos. The question there is not a crawler but a vendor: where does the audio or text go when you use an AI tool, and for how long?

A few habits help:

1. Read what each tool does with your data, and whether it trains on it by default.
2. Prefer tools that let you work on the device when the content is sensitive. We explain how that works in voice typing in 2026, even offline.
3. Keep the choice per task: cloud for convenience, device for confidential material.

This is the model Tiho follows for dictation on Mac and iPhone: you decide whether audio is processed in the cloud at all, and the on-device option keeps speech on your own machine.

The short version

• Four major AI companies back one robots.txt token, __usage_ai-training, to opt out of generative AI training.
• Support is rolling out through 2027, at different dates per company. Keep your old rules until each one confirms.
• It applies going forward, covers training rather than search, and depends on good faith.
• For private work, the real protection is choosing tools that process your words where you want them processed.