Unverified50% confidenceBenchmarkExact time
2024年以前主流模型在SWE-bench上的通过率普遍低于5%，Claude 3.5系列发布后跃升至49%（SWE-bench Verified子集）
1
Sources
50%
Confidence
Long-term
Relevance
7/6/2026
First Seen
Sources
Spotify's AI in Practice: How 3,000 Engineers Are Reinventing R&D with Agents
bilibiliGelai_AI7/3/2026
Related Claims
UnverifiedEarly GPT-4 had a pass rate of only 1.7% on SWE-bench, Claude 3 Opus approximately 4.8%, and it was not until mid-2024 that the 20% mark was surpassed.73% similarUnverifiedClaude Opus 4.6 scored 53.4% on the SWE-Bench Pro benchmark.70% similarVerifiedIn 2024, models such as Claude 3.5 Sonnet have already exceeded 50% on the SWE-bench Verified subset.69% similarUnverifiedOpus 4's pass rate on SWE-bench Verified exceeds 72%, significantly ahead of other competing models.65% similarUnverifiedIn SWE-bench testing, the early GPT-4 resolution rate was below 2%, while Claude 3.5 Sonnet broke through 49%.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/112968API
curl https://kongchang.com/api/v1/knowledge/claims/112968MCP
get_claim(id=112968)