Unverified50% confidenceFactExact time
82.7% of reachable sites have a robots.txt; 25.1% block at least one search or answer crawler, and 10.2% block most of them; 26.0% block at least one training crawler, but only 2.6% block only training crawlers while keeping search and answer crawlers open; 2.5% carry Content-Signal directives, and 62.4% list a sitemap.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/15/2026
First Seen
Valid until: 1/13/2027
Sources
Related Entities
Related Claims
UnverifiedGPTBot's server rejection rate was 14.4%, of which 11.1% occurred while robots.txt allowed it; ClaudeBot was rejected 14.8%, with 10.9% under allowance; PerplexityBot was rejected 11.3%, with 8.4% under allowance; OAI-SearchBot was rejected 9.8%, with 8.0% under allowance.71% similarUnverifiedAgentThreatBench tests show that without guardrails, agents follow planted traps with up to a 27% probability under ordinary instructions, and as high as 80% when explicitly told to obey planted text; with OpenAPPA added, none of the 360 traps worked.59% similarUnverifiedIn scenarios such as customer service bots, FAQ answering, and document summarization, the semantic cache hit rate can reach 30%–60%.58% similarUnverifiedAccording to Google's safety data, unauthorized shortcuts accounted for 2% of coding evaluation attempts, and indirect prompt injection attacks succeeded in 0.7% of 15 adaptive attempts.57% similarUnverifiedAn Anthropic randomized trial had 52 engineers (mostly junior) learn an unfamiliar Python library; the AI-assisted group averaged 50% on the comprehension quiz, while the handwritten-code group averaged 67%, with the largest gap on debugging-type questions; the speed advantage brought by AI was about 2 minutes and was not statistically significant.57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1040871API
curl https://kongchang.com/api/v1/knowledge/claims/1040871MCP
get_claim(id=1040871)