Unverified50% confidenceBenchmarkExact time
When the total was obscured but each line item was visible, Claude and Gemini 3 Flash got all 48 correct, with all failures coming from the smaller models (Flash-Lite failed 31%, nano failed 79%), and nano got only 12/48 correct even when explicitly instructed to add up the items.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/11/2026
First Seen
Valid until: 1/9/2027
Sources
Related Entities
Related Claims
UnverifiedThe median confidence difference across models when fabricating totals is enormous: Gemini 3 Flash and Flash-Lite at 95, GPT-5.4 nano at 62, and Claude Sonnet 5 at only 20.78% similarUnverifiedDeepSeek V4 Flash failed 13 out of 22 multi-turn tests (fail_fast 45%), and Gemini also failed 13.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/997263API
curl https://kongchang.com/api/v1/knowledge/claims/997263MCP
get_claim(id=997263)