Expired90% confidenceFactExact time
SWE-bench uses real GitHub Issues directly as test problems, requiring models to locate and fix bugs within a complete code repository.
21
Sources
90%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
Why Is AI Progressing Fastest in Coding? A Deep Dive into Four Structural Advantages
bilibili卢菁博士_北大AI博士后6/5/2026
Related Claims
VerifiedSWE-bench requires models to locate and fix actual issues in real GitHub repositories.84% similarUnverifiedSWE-bench requires models to autonomously complete code modifications starting from real GitHub issues and pass test cases, making it closer to real-world engineering scenarios than HumanEval.82% similarVerifiedSWE-bench was developed by a Princeton University team; the Pro version requires models to locate bugs and generate fix patches within real GitHub repositories.81% similarUnverifiedSWE-bench requires models to resolve real software defects on GitHub and is the authoritative benchmark for evaluating agent-level programming capabilities.81% similarUnverifiedSWE-bench要求模型在真实GitHub仓库中定位问题并生成可通过CI测试的补丁，以Resolve Rate作为核心指标81% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/53529API
curl https://kongchang.com/api/v1/knowledge/claims/53529MCP
get_claim(id=53529)