Expired60% confidenceBenchmarkExact time
Multiple evaluations targeting SWE-bench in 2024 showed that state-of-the-art AI agents achieved a resolution rate of only 20%–50% on real GitHub Issues.
2
Sources
60%
Confidence
Medium-term (~90 days)
Relevance
7/7/2026
First Seen
Valid until: 10/5/2026(expired)
Sources
The Short Leash Method: A Practical Methodology for Staying in Control of AI-Assisted Programming
hackernewshackernews7/2/2026
Related Claims
UnverifiedSWE-bench requires models to resolve real GitHub issues, and the pass rate does not exceed 50% even for the strongest models.77% similarUnverifiedThe SWE-bench benchmark from Princeton University in 2023 showed that the most advanced AI programming tools at the time achieved a success rate of only about 13% on real GitHub issue fix tasks.77% similarUnverified根据SWE-RPG基准论文数据，当前顶级编程代理在真实GitHub问题上的平均解决率仅为31.5%77% similarUnverifiedTop-tier code agents are already capable of autonomously resolving more than 50% of real GitHub issues.75% similarUnverifiedSWE-bench requires models to locate and fix bugs in real GitHub repositories, with the pass rate gradually increasing from less than 2% initially to over 20%.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/196717API
curl https://kongchang.com/api/v1/knowledge/claims/196717MCP
get_claim(id=196717)