Unverified50% confidenceFactExact time
By re-tallying the results files, readers found a row of data that the authors had skipped: the maintainer file lists 9 of 212 tests judged WEAK (4%), while the agent file lists 79 of 591 (13%).
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn the corrected data, counted by change, 14 of 91 agent PRs contain WEAK tests, while only 1 of 71 maintainer commits does.73% similarUnverifiedThis benchmark uses 10 original short documents of 200 to 250 words each, with 18 planted anomalies, 37 scoring entries in total (18 dirty and 19 clean), testing 13 models with 74 prompts per model.71% similarUnverifiedThe author re-rated the 112 items based on the code's actual capabilities: 29 ✅, 64 🟡, 19 ⬜, versus the published status of 43/52/17, with 16 items rated by the author lower than the coverage status the tool publicly claims.69% similarUnverified19.4% (6,251 articles) of duplicate document contamination exists in the naive test set split.68% similarUnverifiedAmong the approximately 2,000 papers examined, more than 1,100 had at least one key claim independently verified, of which 266 were fully reproduced, 632 were partially reproduced through small-scale experiments, and approximately 4,000 individual claims were confirmed.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1028487API
curl https://kongchang.com/api/v1/knowledge/claims/1028487MCP
get_claim(id=1028487)