Unverified50% confidenceBenchmarkExact time
Among the 95 tasks, only 14 tasks were truly solved by the best configuration (score no lower than 0.8 and no incidents).
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/16/2026
First Seen
Valid until: 1/14/2027
Sources
Related Entities
Related Claims
UnverifiedThe sealed task pack contains 30 tasks, divided into 20 ordinary tasks and 10 negative control tasks.64% similarUnverifiedTask completion rate is defined as the proportion of given programming tasks completed without human intervention, and better reflects real-world usability than traditional metrics such as BLEU scores.62% similarUnverifiedAfter RL training, the small model successfully completes 16 out of 24 tasks, whereas before RL it could not successfully complete any tasks.61% similarUnverifiedThe source material describes a task broken into 11 nodes that ultimately all passed validation.61% similarUnverifiedThe benchmark consists of 12 real tasks (two per category), all scored deterministically, completed on October 11, 2026, covering 10 models, with one attempt per task, at a total cost of $1.78.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1041818API
curl https://kongchang.com/api/v1/knowledge/claims/1041818MCP
get_claim(id=1041818)