Unverified50% confidenceBenchmarkExact time
The numbers task was the hardest for 6 of the 8 models, with scores on large numbers (10¹⁵ to 10³⁹) ranging from only 0.06 to 0.53.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/11/2026
First Seen
Valid until: 1/9/2027
Sources
Related Entities
Related Claims
UnverifiedA WANDR score of 0.610 does not mean the model fully succeeds on 60% of tasks; the actual partial completion rate is related to the distribution of task complexity.68% similarUnverifiedThe model has 512 experts per layer, but only 10 are activated per token, meaning the weights actually participating in computation are approximately 6B.65% similarUnverifiedRanked by question difficulty, persistence rate was 91.4% on the 60 questions that all models answered correctly, but only 73.5% on the 52 questions that at least one model got wrong, indicating that sycophancy tracks uncertainty.65% similarUnverifiedAmong the ten environments tested in the paper, seven could not apply statistical synthesizers due to hard requirements for seed data.64% similarUnverifiedThe user pointed out that these models achieve high scores on benchmarks yet frequently fail at the simplest tasks.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/999203API
curl https://kongchang.com/api/v1/knowledge/claims/999203MCP
get_claim(id=999203)