Unverified50% confidenceFactExact time
The Meta model card footnote explains: the score selection method prioritizes comparable third-party results, otherwise uses the highest of internal validation runs, self-reported scores, or official leaderboard scores; competitor runs are best-effort and not optimized for proprietary models; the SWE-Marathon comparison involves different agent harnesses; Wonder Full's 57.5 on SWE-Marathon is reported with xhigh reasoning effort.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedMeta admits that its internally run competitor tests "use common settings where feasible, but are still best-effort: prompts, tools, and runtime may not be optimized for proprietary models".75% similarUnverifiedEvaluating model capabilities should combine third-party benchmark results, long-term real user feedback, and the effectiveness of deployment in specific business scenarios, rather than relying solely on version numbers.68% similarUnverified基准对比中竞品使用的是自适应推理（Adaptive Reasoning）模式，而非强制开启最高推理档位66% similarUnverified选择和使用不同模型档位时不能仅凭版本号判断，基于真实场景的多轮测试才是评估模型能力的可靠依据66% similarUnverifiedCherry-picking runs refers to selecting and reporting the highest score from multiple runs on the same questions, which measures the upper bound of the model's luck rather than stable expected performance.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1011258API
curl https://kongchang.com/api/v1/knowledge/claims/1011258MCP
get_claim(id=1011258)