ChatGPT-User
A crawler operated by OpenAI that handles real-time user requests in ChatGPT, for example triggering a web page visit within a user conversation.
Core Facts
OpenAI's official documentation states that settings such as OAI-SearchBot, GPTBot, and ChatGPT-User are independent of one another; OAI-SearchBot is responsible for making sites appear in ChatGPT search, and sites that opt out will not appear in ChatGPT search answers.
AI crawlers include GPTBot, ClaudeBot, PerplexityBot, and others.
Timeline (last 90 days)
If retrieval-type AI crawlers (such as OAI-SearchBot and ChatGPT-User) are blocked, the content will not be cited in AI answers; but if only training-type crawlers (such as GPTBot) are blocked, the content may still be cited.
GPTBot and Google-Extended are training-type crawlers, while OAI-SearchBot and ChatGPT-User are retrieval-type crawlers used to read content when answering queries in real time.
A misspelling in the user-agent name will cause the rule to fail silently; the correct names include GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, and Perplexity-User.
The opt-out cost for the three types of AI crawlers forms a gradient: training crawlers < search crawlers < user-triggered fetchers
Anthropic says that disallowing Claude-User will prevent Claude from retrieving pages for user queries, but OpenAI and Perplexity say their own user-triggered fetchers may not obey robots.txt
AI crawler user agents can be divided into three categories: training (GPTBot, ClaudeBot, CCBot, Google-Extended, Applebot-Extended), search indexing (OAI-SearchBot, Claude-SearchBot, PerplexityBot), and user-triggered (ChatGPT-User, Claude-User, Perplexity-User).
OpenAI's official documentation states that settings such as OAI-SearchBot, GPTBot, and ChatGPT-User are independent of one another; OAI-SearchBot is responsible for making sites appear in ChatGPT search, and sites that opt out will not appear in ChatGPT search answers.
Marketers should check CDN or WAF logs to confirm that OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot and Googlebot receive 200 responses rather than 403.
The access permissions set in robots.txt for AI crawlers such as GPTBot, ClaudeBot, and PerplexityBot directly determine whether content can enter these models' training sets or real-time retrieval pools
All Facts (10)
OpenAI's official documentation states that settings such as OAI-SearchBot, GPTBot, and ChatGPT-User are independent of one another; OAI-SearchBot is responsible for making sites appear in ChatGPT search, and sites that opt out will not appear in ChatGPT search answers.
80%VerifiedAI crawlers include GPTBot, ClaudeBot, PerplexityBot, and others.
65%UnverifiedIf retrieval-type AI crawlers (such as OAI-SearchBot and ChatGPT-User) are blocked, the content will not be cited in AI answers; but if only training-type crawlers (such as GPTBot) are blocked, the content may still be cited.
50%UnverifiedGPTBot and Google-Extended are training-type crawlers, while OAI-SearchBot and ChatGPT-User are retrieval-type crawlers used to read content when answering queries in real time.
50%UnverifiedA misspelling in the user-agent name will cause the rule to fail silently; the correct names include GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, and Perplexity-User.
50%UnverifiedThe opt-out cost for the three types of AI crawlers forms a gradient: training crawlers < search crawlers < user-triggered fetchers
50%UnverifiedAnthropic says that disallowing Claude-User will prevent Claude from retrieving pages for user queries, but OpenAI and Perplexity say their own user-triggered fetchers may not obey robots.txt
50%UnverifiedAI crawler user agents can be divided into three categories: training (GPTBot, ClaudeBot, CCBot, Google-Extended, Applebot-Extended), search indexing (OAI-SearchBot, Claude-SearchBot, PerplexityBot), and user-triggered (ChatGPT-User, Claude-User, Perplexity-User).
50%UnverifiedMarketers should check CDN or WAF logs to confirm that OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot and Googlebot receive 200 responses rather than 403.
50%UnverifiedThe access permissions set in robots.txt for AI crawlers such as GPTBot, ClaudeBot, and PerplexityBot directly determine whether content can enter these models' training sets or real-time retrieval pools
50%