Unverified50% confidenceOpinionExact time
There is a qualitative gap between the currently popular evaluation benchmarks GSM8K and MATH and frontier research-level mathematical reasoning
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/9/2026
First Seen
Valid until: 11/7/2026
Sources
AI反例被证伪：Connes刚性定理事件揭示的数学推理真相
Related Claims
VerifiedGSM8K是小学数学推理基准，HumanEval是代码生成基准，HellaSwag是常识推理基准75% similarUnverifiedProcess Reward Models (PRM) have significantly improved model reasoning accuracy on mathematical reasoning benchmarks such as GSM8K.70% similarUnverified在代码语料比例更高的混合数据集上训练的模型，在数学推理基准（如MATH、GSM8K）上的表现显著优于纯自然语言训练的同规模模型，这一结论获斯坦福大学和MIT的多项研究证实69% similarUnverified2022年Wei等人的论文《Chain-of-Thought Prompting Elicits Reasoning in Large Language Models》证明在few-shot示例中加入推理步骤可将PaLM 540B在GSM8K上的准确率从17.9%提升至58.1%68% similarVerifiedThe Chain-of-Thought prompting paper published in 2022 by Jason Wei et al. of Google Brain was the first to systematically demonstrate that adding step-by-step reasoning examples significantly improves the accuracy of large models such as PaLM on mathematical reasoning tasks66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/713772API
curl https://kongchang.com/api/v1/knowledge/claims/713772MCP
get_claim(id=713772)