Unverified50% confidenceBenchmarkExact time
All incorrect distractor counting answers were without exception overestimates, and the token counts of incorrect answers were clearly higher than those of correct answers: for gpt-5.4-mini at high, the median token count of incorrect answers was 412 vs 91.5 for correct answers; for gpt-oss-120b, incorrect answers were 5,776 tokens vs 747 tokens for correct answers.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedThe parsing defect combined with one incorrect answer key caused gpt-oss's score across five core tasks to shift from 0.62 to 1.00, a shift as large as 0.38.69% similarUnverifiedGPT-5.5 ran out of usage quota, leaving 35 of the 90 questions unfinished; all 55 answered questions were correct, but Kaggle counted the unanswered questions as wrong, causing its official score to be lower.69% similarUnverifiedMistral Small 4 scored only 0.699 on unit conversion inputs and produced 265 errors that differed from the correct label by exactly a power of ten.67% similarUnverifiedIn the first run, Claude Opus 5 scored only 0.08 and GPT-5.5 errored out entirely; upon investigation this was found to be a 403 PermissionDeniedError quota-exceeded error, where concurrent requests burned through the quota for expensive models and the scorer incorrectly treated errored calls as wrong answers.67% similarUnverifiedClaude Opus 4.6 produced 30 times more errors than GPT-5.4, averaging 2.5 errors per trajectory vs. 0.08, with the difference being statistically significant (p < 0.0001, Cliff's δ = 0.613).66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1007699API
curl https://kongchang.com/api/v1/knowledge/claims/1007699MCP
get_claim(id=1007699)