Unverified50% confidenceFactExact time
Mainstream AI crawlers such as GPTBot, ClaudeBot and PerplexityBot currently largely do not execute JavaScript, so for sites that depend on JavaScript rendering, even if both robots.txt and the WAF let them through, the content they get is still empty.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedCrawlers for AI systems such as ChatGPT and Perplexity mostly do not execute JavaScript, so if page content depends on JS rendering, they may see a blank page; although Googlebot can render JS, there are delays and quota limits.77% similarUnverifiedAI爬虫（如GPTBot、ClaudeBot、Common Crawl）采用轻量级HTTP请求+HTML解析方式，不为每个页面启动完整浏览器执行JavaScript67% similarUnverifiedWhat a crawler retrieves is only unrendered HTML source code, because crawlers lack the rendering capabilities of a browser.66% similarUnverifiedAI crawlers only download HTML and do not execute JavaScript, so website analytics tools that rely on browser-side JavaScript cannot count AI crawler visits.64% similarUnverified无头Chrome自Chrome 59版本（2017）起原生支持，可配合Puppeteer或Playwright等控制库执行JavaScript、管理Session并模拟浏览器指纹63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1040875API
curl https://kongchang.com/api/v1/knowledge/claims/1040875MCP
get_claim(id=1040875)