Unverified50% confidenceBenchmarkExact time
On a held-out test set of 1,200 problems, the calculation scores of 12 models range from 33.3% to 100.0%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/4/2026
First Seen
Valid until: 2/2/2027
Sources
Related Entities
Related Claims
UnverifiedThis benchmark uses 10 original short documents of 200 to 250 words each, with 18 planted anomalies, 37 scoring entries in total (18 dirty and 19 clean), testing 13 models with 74 prompts per model.76% similarUnverifiedThe accuracy data in the test has an error margin of approximately ±8–10 percentage points at n=100 sample size (based on a 95% confidence-level binomial confidence interval).75% similarUnverifiedIn the experiments, the task resolution rate was 17% for zero-shot generation, 27% for generic self-correction, 23% for error-only feedback, and 90% for diagnostic counterexample feedback (A-CEGIS).75% similarUnverifiedDuring testing, cases emerged where a model claimed 95% confidence in a move but scored 0 points according to the evaluation criteria.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1134261API
curl https://kongchang.com/api/v1/knowledge/claims/1134261MCP
get_claim(id=1134261)