The four leading AI developers will recognize a single policy token in a site’s robots.txt file, making it easier for publishers to block their content from AI training.

The four biggest AI developers—OpenAI, Anthropic, Microsoft, and Google—have agreed to honor a single opt‑out token in a site’s robots.txt file, simplifying how publishers can block their content from AI training.

What the unified token means

When a website includes the token "AI‑OPT‑OUT" in its robots.txt, each of the four companies will automatically exclude that site’s data from future model training, reducing the need for multiple, vendor‑specific directives.

How the agreement was reached

Representatives from the four firms met at the AI Ethics Summit in Berlin last month and signed a memorandum of understanding that standardizes the opt‑out mechanism across their platforms, including ChatGPT, Claude, Bing, and Gemini.

The move follows growing pressure from publishers, regulators, and privacy advocates who argue that the current patchwork of policies is confusing and often ignored.

Implications for content creators

Publishers can now add a single line to their robots.txt—e.g., User-agent: * Disallow: / AI-OPT-OUT: true—and expect consistent compliance from the four AI giants.

  • Simplifies compliance for webmasters
  • Reduces legal uncertainty around data usage
  • Sets a precedent for other AI developers to follow

While the token covers the four signatories, other AI companies are encouraged to adopt the same standard, potentially creating an industry‑wide baseline for data protection.

For more context on the broader AI policy landscape, see the Technology section on uxc.news.

Learn about OpenAI’s stance on data privacy on Momentum and Anthropic’s approach on Momentum.

The agreement was first reported by Dev.to’s AI Daily Digest.