Unverified50% confidenceBenchmarkExact time
On the SimpleQA benchmark, o4-mini declines to answer only 1% of questions, with an error rate of 75%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/18/2026
First Seen
Valid until: 1/16/2027
Sources
Related Entities
Related Claims
Unverifiedo4-mini and gpt-5-thinking-mini have almost the same accuracy on SimpleQA, but the latter makes roughly three times fewer errors than the former.74% similarUnverifiedOn the SimpleQA benchmark, gpt-5-thinking-mini declines to answer 52% of questions, with an error rate of 26%.74% similarUnverifiedQwen Next scored only 6/108 (5.56%) in testing due to format reliability issues (17 invalid_json responses, with only 6 entries scorable) rather than insufficient comprehension.72% similarUnverifiedMistral Small 4 scored only 0.699 on unit conversion inputs and produced 265 errors that differed from the correct label by exactly a power of ten.66% similarUnverifiedAfter testing, programming blogger Theo pointed out that more than half of Gemini 4's judgments about his codebase were wrong, making it the only one of the 7 tested models with more than half errors.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1063181API
curl https://kongchang.com/api/v1/knowledge/claims/1063181MCP
get_claim(id=1063181)