Unverified60% confidenceBenchmarkExact time
Early GPT-4 had a pass rate of only 1.7% on SWE-bench, Claude 3 Opus approximately 4.8%, and it was not until mid-2024 that the 20% mark was surpassed.
2
Sources
60%
Confidence
Long-term
Relevance
7/8/2026
First Seen
Sources
Claude Sonnet 5接入Devin：编程性能升级，配额消耗降低30%
rss6/30/2026
Related Claims
Unverified在SWE-bench最初公开结果中GPT-4的解决率仅约1.7%，Devin宣称达到13.86%，SWE-agent达到12.5%76% similarUnverified2024年以前主流模型在SWE-bench上的通过率普遍低于5%，Claude 3.5系列发布后跃升至49%（SWE-bench Verified子集）73% similarUnverifiedIn Automation Bench at medium reasoning intensity, GPT-6.1 Sol scored 35.4%, which is 2.2 percentage points higher than Opus 5.5 at one-third of the cost.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/212301API
curl https://kongchang.com/api/v1/knowledge/claims/212301MCP
get_claim(id=212301)