Expired60% confidenceBenchmarkExact time
The pass rate on the SWE-bench Verified subset jumped from under 5% in early 2024 to 49%.
2
Sources
60%
Confidence
Medium-term (~90 days)
Relevance
7/6/2026
First Seen
Valid until: 10/4/2026(expired)
Sources
Spotify's AI in Practice: How 3,000 Engineers Are Reinventing R&D with Agents
bilibiliGelai_AI7/3/2026
Related Claims
UnverifiedThe solve rate of top models on SWE-bench jumped from under 5% in 2023 to over 30% in 2025.75% similarUnverifiedIn early 2023, mainstream models' SWE-bench scores were generally below 5%; by mid-2025, top models had surpassed 70%.75% similarUnverifiedIn 2024, some models' success rates on SWE-bench rose to the 30%–50% range.74% similarVerifiedIn 2024, top models achieved a pass rate of approximately 50–70% on SWEbench-Verified.73% similarUnverifiedOn SWE-bench, early models had a solve rate below 5%, while mainstream products in 2025 have surpassed 50%.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/113217API
curl https://kongchang.com/api/v1/knowledge/claims/113217MCP
get_claim(id=113217)