From 3918cc9ea8a7604e9de1d7826816b01d3d836d6a Mon Sep 17 00:00:00 2001 From: Elie Habib Date: Thu, 23 Apr 2026 09:15:43 +0400 Subject: [PATCH] feat(agent-readiness): declare Content-Signal in robots.txt (#3327) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per contentsignals.org draft RFC: declare AI content-usage preferences at the origin level. Closes #3307, part of epic #3306. Values: ai-train=no — no consent for model-training corpora search=yes — allow search indexing for referral traffic ai-input=yes — allow live agent retrieval (Perplexity, ChatGPT browsing, Claude, etc.) --- public/robots.txt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/public/robots.txt b/public/robots.txt index 280151770..49ad60992 100644 --- a/public/robots.txt +++ b/public/robots.txt @@ -1,4 +1,9 @@ # World Monitor - protect API routes from crawlers + +# AI content-usage preferences (contentsignals.org draft RFC). +# Origin-wide directive, NOT scoped under User-agent per the spec. +Content-Signal: ai-train=no, search=yes, ai-input=yes + User-agent: * Allow: / Allow: /api/story