Expired50% confidenceBenchmarkExact time
The resolution rate of the best-performing systems on the SWE-bench benchmark has risen from less than 5% initially to over 40%.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
8/18/2026
First Seen
Valid until: 9/1/2026(expired)
Sources
Related Claims
UnverifiedWhen SWE-bench was first released, the pass rate of top models was below 2%; the pass rate of leading systems has now surpassed 50%.78% similarUnverifiedThe pass rate on the SWE-bench Verified subset jumped from under 5% in early 2024 to 49%.73% similarUnverifiedSWE-bench is a benchmark measuring models' ability to autonomously fix real GitHub Issues; early models had a resolution rate below 5%, and by 2025 mainstream products have surpassed 50%.70% similarUnverifiedOn SWE-bench, early models had a solve rate below 5%, while mainstream products in 2025 have surpassed 50%.68% similarUnverified在SWE-bench Full版本上，多数模型的通过率仍低于20%68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/769922API
curl https://kongchang.com/api/v1/knowledge/claims/769922MCP
get_claim(id=769922)