Unverified50% confidenceBenchmarkExact time
SWE-bench requires models to locate and fix bugs in real GitHub repositories, with the pass rate gradually increasing from less than 2% initially to over 20%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/4/2026
First Seen
Valid until: 11/2/2026
Sources
Related Claims
UnverifiedSWE-bench requires models to resolve real GitHub issues, and the pass rate does not exceed 50% even for the strongest models.79% similarUnverifiedIn the SWE-bench real GitHub Issue fix task, the fix success rates of models from different vendors can differ by 15 to 30 percentage points.77% similarUnverifiedSWE-bench要求模型在真实GitHub仓库中定位问题并生成可通过CI测试的补丁，以Resolve Rate作为核心指标76% similarVerifiedSWE-bench requires models to locate and fix actual issues in real GitHub repositories.76% similarVerifiedSWE-bench uses real GitHub Issues directly as test problems, requiring models to locate and fix bugs within a complete code repository.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/684192API
curl https://kongchang.com/api/v1/knowledge/claims/684192MCP
get_claim(id=684192)