Perplexity-User
Perplexity's real-time user agent crawler. When a Perplexity user requests to browse a web page, it fetches the page content on that user's behalf.
Timeline (last 90 days)
Anthropic says that disallowing Claude-User will prevent Claude from retrieving pages for user queries, but OpenAI and Perplexity say their own user-triggered fetchers may not obey robots.txt
AI crawler user agents can be divided into three categories: training (GPTBot, ClaudeBot, CCBot, Google-Extended, Applebot-Extended), search indexing (OAI-SearchBot, Claude-SearchBot, PerplexityBot), and user-triggered (ChatGPT-User, Claude-User, Perplexity-User).
All Facts (2)
Anthropic says that disallowing Claude-User will prevent Claude from retrieving pages for user queries, but OpenAI and Perplexity say their own user-triggered fetchers may not obey robots.txt
50%UnverifiedAI crawler user agents can be divided into three categories: training (GPTBot, ClaudeBot, CCBot, Google-Extended, Applebot-Extended), search indexing (OAI-SearchBot, Claude-SearchBot, PerplexityBot), and user-triggered (ChatGPT-User, Claude-User, Perplexity-User).
50%