Unverified50% confidenceFactExact time
The robots.txt protocol was originally established in 1994 as an industry convention by which websites inform crawlers which pages may be scraped, but it carries no legally binding force.
1
Sources
50%
Confidence
Long-term
Relevance
10/1/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe robots.txt protocol does not carry legally binding enforcement power.72% similarUnverifiedSome publishers have begun blocking AI crawlers in the robots.txt protocol, but this technical specification carries no legal enforcement power and its implementation depends on voluntary compliance by AI companies.67% similarUnverified《纽约时报》、路透社等大型媒体在robots.txt中屏蔽AI爬虫，并通过法律诉讼主张内容权益57% similarUnverifiedThe X platform restricts AI crawlers' access in its robots.txt and takes legal action against unauthorized data scraping.57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/966095API
curl https://kongchang.com/api/v1/knowledge/claims/966095MCP
get_claim(id=966095)