Unverified50% confidenceBenchmarkExact time
SWE-bench is based on 2,294 GitHub issues from 12 real open-source projects, and the highest pass rate achieved by current systems is only around 50%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/4/2026
First Seen
Valid until: 11/2/2026
Sources
Claude Opus 5登陆Devin：性能媲美Fable成本减半
Related Claims
UnverifiedSWE-bench requires models to resolve real GitHub issues, and the pass rate does not exceed 50% even for the strongest models.75% similarUnverifiedSWE-bench requires models to locate and fix bugs in real GitHub repositories, with the pass rate gradually increasing from less than 2% initially to over 20%.73% similarUnverifiedAccording to GitHub's Octoverse report, more than 60% of code review work in the top 1% of open-source projects is concentrated among fewer than 3 maintainers.73% similarUnverifiedSWE-bench was released by Princeton's NLP team in 2024 and contains 2,294 real GitHub Issues from 12 mainstream open-source projects, with modifications spanning an average of 3.7 files.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/685070API
curl https://kongchang.com/api/v1/knowledge/claims/685070MCP
get_claim(id=685070)