Google, Microsoft, OpenAI Share AI Training Opt-Out Signal

Google, Microsoft, and OpenAI announced on October 30 that they will support a shared robots.txt signal letting multiple AI companies know at once that a site opts out of AI training, Search Engine Journal reported on November 1. The signal works by placing the value '__usage_ai-training' in the user-agent field and writing Disallow and Allow rules beneath it, reusing existing robots.txt syntax rather than adding a new directive. The signal doesn't block crawling itself; it only communicates that content shouldn't be used for AI training.

The signal originated from a draft Kevin Kelly, an Anthropic employee, submitted to the IETF AI Preferences working group, and it hasn't yet been adopted as an official IETF standard. None of the three companies currently honor the signal, and each said it will begin applying it at a different time. Anthropic itself isn't among the three companies named in the announcement; Kelly said Anthropic has also started internal work on it, but rolling it out across the full service will take time.

Google said it plans to support the signal starting in early 2027, and that if a site uses only this signal without separately mentioning Google-Extended, Google will interpret it as opting out of Google-Extended as well. The rule will also apply to grounding in the Gemini app, while AI Overviews and AI Mode will keep using Search Console's separate generative AI visibility settings. OpenAI said it will support the signal within a few months, and until then its existing GPTBot policies remain unchanged.

Microsoft chose to manage training and grounding settings separately. Starting December 15, a temporary grounding setting will be added to Bing Webmaster Tools for sites that have relied on noarchive or nocache tags; the new setting actually takes effect starting February 1 next year, and by March 1 the old tags will no longer be recognized. Pages with no setting applied before then stay open to grounding until they're next crawled.

None of the three companies currently honor this shared signal, so there's no need to remove longstanding per-company rules like Google-Extended, GPTBot, or ClaudeBot. Adding the '__usage_ai-training' line to robots.txt now costs nothing, but it's safer to keep it alongside existing rules and watch each company's rollout date: Google in early 2027, OpenAI within a few months, Microsoft on February 1 next year. Sites using Bing Webmaster Tools with noarchive or nocache tags in place should pay particular attention to the December 15 update and the March 1 deadline next year.

Source: Search Engine Journal, November 1, 2026. Checked November 2, 2026. Read the original