Verified65% confidenceFactExact time
SWE-bench requires models to fix issues in real GitHub repositories, while HumanEval/MBPP are function-level code generation benchmarks.
3
Sources
65%
Confidence
Long-term
Relevance
8/7/2026
First Seen
Sources
Qwen3.8-Max发布：2.4万亿参数自主编程新标杆
Related Claims
VerifiedSWE-bench requires models to locate and fix actual issues in real GitHub repositories.80% similarVerifiedSWE-bench uses real GitHub Issues directly as test problems, requiring models to locate and fix bugs within a complete code repository.78% similarVerifiedSWE-bench was developed by a Princeton University team; the Pro version requires models to locate bugs and generate fix patches within real GitHub repositories.76% similarUnverifiedMMLU考察跨学科知识理解、HumanEval测试代码生成能力、SWE-bench专门针对真实GitHub Issue修复能力74% similarUnverifiedSWE-bench要求模型在真实GitHub仓库中定位问题并生成可通过CI测试的补丁，以Resolve Rate作为核心指标74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/699296API
curl https://kongchang.com/api/v1/knowledge/claims/699296MCP
get_claim(id=699296)