Unverified50% confidenceFactExact time

Anthropic says that disallowing Claude-User will prevent Claude from retrieving pages for user queries, but OpenAI and Perplexity say their own user-triggered fetchers may not obey robots.txt

1

Sources

50%

Confidence

Medium-term (~90 days)

Relevance

10/14/2026

First Seen

Valid until: 1/12/2027

Sources

AI Crawlers Explained: Which Ones to Block and Which to Allow

Related Entities

Claude-Userproduct ChatGPT-Userproduct Perplexity-Userproduct Anthropiccompany OpenAIcompany Perplexitycompany

Related Claims

UnverifiedAnthropic prohibits users from using Claude's output to train competing models.62% similar UnverifiedThe web version of Claude can only generate text; it cannot proactively touch the user's file system or verify whether generated code can actually run.58% similar UnverifiedCowork cannot force web search and often falls back to an unreliable browser extension, whereas Claude Chat can force, enable, or disable web search and switch mid-conversation.58% similar

Cite This Claim

Stable URI

https://kongchang.com/claim/1028531

API

curl https://kongchang.com/api/v1/knowledge/claims/1028531

MCP

get_claim(id=1028531)