## [AI Policy Token: the IETF robots.txt Opt-Out Standard](/blog/ai-policy-token-robots-txt.md)

October 31, 2026 ·

<!-- -->

5 min read

[NGPilot](https://ngpilot.com)

On October 30, a one-day-old IETF draft became the most consequential robots.txt news since Google, Bing and Yahoo unified crawler rules in 2022. `draft-kelley-aipref-training-preference-signal-00`, authored by Anthropic standards lead Kevin Kelley, defines a **policy token**: a way to tell every major AI crawler at once whether your content may be used for model training — with one `robots.txt` group instead of a dozen per-vendor blocks.

Press reports (The Verge, Search Engine Roundtable) say OpenAI, Google, Microsoft and Anthropic have announced they will adopt the mechanism. Meta voiced support without committing. If your `robots.txt` still names AI crawlers one by one, this draft is the exit from that treadmill.

**Tags:**

* [seo](/blog/tags/seo.md)
* [ai-crawlers](/blog/tags/ai-crawlers.md)
* [robots-txt](/blog/tags/robots-txt.md)
* [geo](/blog/tags/geo.md)
* [ietf](/blog/tags/ietf.md)

[**Read more**](/blog/ai-policy-token-robots-txt.md)