Unverified50% confidenceFactExact time
In the early test rounds, 101 of 122 attempts contained syntax errors and could not run at all.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/19/2026
First Seen
Valid until: 1/17/2027
Sources
Related Entities
Related Claims
UnverifiedThe substantive threshold was validated on the 128 attempts from the filtering round: 121 passed, and the 7 rejected were 6 empty submissions and 1 bare SELECT, with no real attempt lost.67% similarUnverifiedThe ThinkingBox benchmark conducted a total of 121,680 trials, of which 79,853 failed.66% similarUnverifiedIn a verbatim-quotation benchmark spanning 13 models, 2 instruction variants, and 962 scored responses, not a single model corrected any planted error within a quotation.66% similarUnverifiedThe researchers confirmed 91 failures through manual verification and categorized them into 7 failure loci.66% similarUnverifiedIn a scenario with 1,700 easy cases and 100 difficult multilingual cases, even if half of the difficult cases fail entirely, the aggregate pass rate may only drop from 97% to around 93%65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1069229API
curl https://kongchang.com/api/v1/knowledge/claims/1069229MCP
get_claim(id=1069229)