Unverified50% confidenceBenchmarkExact time
Between 2024 and 2025, the solve rates of models such as Claude and Gemini on the SWE-bench software engineering benchmark skyrocketed from under 5% to over 50%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/11/2026
First Seen
Valid until: 12/10/2026
Sources
Related Entities
Related Claims
UnverifiedThe solve rate of top models on SWE-bench jumped from under 5% in 2023 to over 30% in 2025.74% similarUnverifiedOn SWE-bench, early models had a solve rate below 5%, while mainstream products in 2025 have surpassed 50%.72% similarUnverifiedIn early 2023, mainstream models' SWE-bench scores were generally below 5%; by mid-2025, top models had surpassed 70%.71% similarVerifiedIn 2024, top models achieved a pass rate of approximately 50–70% on SWEbench-Verified.69% similarUnverifiedSince 2024, the scores of various AI programming tools on SWE-bench have surged from initially less than 5% to over 50%.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/896130API
curl https://kongchang.com/api/v1/knowledge/claims/896130MCP
get_claim(id=896130)