Unverified50% confidenceBenchmarkExact time
Top paid models such as Claude 3.5 Sonnet and GPT-4o generally achieve resolution rates on SWE-bench that are 10 to 30 percentage points higher than free or lightweight models.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/25/2026
First Seen
Valid until: 12/24/2026
Sources
Related Entities
Related Claims
UnverifiedIn SWE-bench testing, the early GPT-4 resolution rate was below 2%, while Claude 3.5 Sonnet broke through 49%.75% similarUnverified2024-2025年间，Claude 3.5 Sonnet、GPT-4o、Gemini 2.0等模型在HumanEval、SWE-bench等编程基准测试的通过率从不足50%提升至90%以上74% similarUnverifiedThe gap in resolution rate on SWE-Bench Verified between Claude Sonnet 4.6 and GPT-5.1 Codex is only 3.3 percentage points.73% similarUnverifiedGPT-5.6's Extreme High tier achieves a score of 53.6% for approximately $63, whereas Claude Sonnet in adaptive mode spends approximately $2,000 to achieve a score of only 40.5%.72% similarUnverifiedOpus-tier pricing is typically 3-5x that of the mid-tier Sonnet model.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/949813API
curl https://kongchang.com/api/v1/knowledge/claims/949813MCP
get_claim(id=949813)