Expired60% confidenceBenchmarkExact time
The solve rate of top models on SWE-bench jumped from under 5% in 2023 to over 30% in 2025.
2
Sources
60%
Confidence
Medium-term (~90 days)
Relevance
7/9/2026
First Seen
Valid until: 10/7/2026(expired)
Sources
Understanding the Limits of Vibe Coding: A Practical Guide to AI Engineering-Grade Programming
bilibili图灵学院诸葛老师7/8/2026
Related Claims
UnverifiedIn 2024, some models' success rates on SWE-bench rose to the 30%–50% range.84% similarUnverifiedIn early 2023, mainstream models' SWE-bench scores were generally below 5%; by mid-2025, top models had surpassed 70%.80% similarUnverifiedBetween 2024 and 2025, the solve rates of models such as Claude and Gemini on the SWE-bench software engineering benchmark skyrocketed from under 5% to over 50%.74% similarVerifiedIn 2024, top models achieved a pass rate of approximately 50–70% on SWEbench-Verified.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/378716API
curl https://kongchang.com/api/v1/knowledge/claims/378716MCP
get_claim(id=378716)