Unverified50% confidenceBenchmarkExact time
In a verbatim-quotation benchmark spanning 13 models, 2 instruction variants, and 962 scored responses, not a single model corrected any planted error within a quotation.
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn memory recall tests, the 12B model answered 82 out of 100 questions correctly and the 31B model answered 98 out of 100, with neither model hallucinating answers.69% similarUnverifiedThe author blind-checked all 12 draft answer keys using three models not involved in scoring, retaining only those agreed upon by at least two models, which resulted in 2 rows being deleted.68% similarUnverifiedModels' standard accuracy on the passage-level ranking task was only 0% to 13%.65% similarUnverifiedTraditional benchmarks have become ineffective for evaluating model reasoning ability because almost every model may have seen the standard answers in its training data.65% similarUnverifiedThis benchmark uses 10 original short documents of 200 to 250 words each, with 18 planted anomalies, 37 scoring entries in total (18 dirty and 19 clean), testing 13 models with 74 prompts per model.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1000249API
curl https://kongchang.com/api/v1/knowledge/claims/1000249MCP
get_claim(id=1000249)