Unverified50% confidenceBenchmarkExact time
The best-performing largest model can solve 79.6% of held-out test set problems, but only 55.3% on TranSGrid, dropping sharply to 15.8% on the hardest subset.
1
Sources
50%
Confidence
Long-term
Relevance
9/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFor the 7B model, accuracy with the solver was 75.31% versus 72.02% for direct calculation, a paired improvement of +3.29 percentage points, with a 95% cluster interval of [-3.49, 10.38] crossing zero, making the advantage non-significant.73% similarUnverifiedA benchmark score of 73.4% means that approximately one quarter of tasks still fall short, and this score was achieved under maximum compute.70% similarUnverified根据SWE-RPG基准论文，46.7%的代理失败发生在规划和需求分析阶段66% similarVerified80% of the success or failure of fine-tuning depends on data quality, and data preparation typically accounts for 60%–70% of the total project workload.66% similarVerifiedIn long-chain tasks, if each step has an error rate of approximately 5%, the overall success rate may drop to below 36% after 20 steps.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/931095API
curl https://kongchang.com/api/v1/knowledge/claims/931095MCP
get_claim(id=931095)