Unverified50% confidenceBenchmarkExact time
在SWE-bench最初公开结果中GPT-4的解决率仅约1.7%，Devin宣称达到13.86%，SWE-agent达到12.5%
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/22/2026
First Seen
Valid until: 10/20/2026
Sources
Related Claims
Unverified早期GPT-4在SWE-bench的通过率仅为1.7%，Claude 3 Opus约为4.8%，直到2024年中才突破20%76% similarUnverified在 SWE-bench 测试中，早期 GPT-4 解决率不足 2%，而 Claude 3.5 Sonnet 突破 49%75% similarUnverifiedOpus 4.7每次解题的中位输出token数为60,000，而GPT 5.5仅需16,00070% similarUnverified在GPT-4发布初期，主流模型在GPQA上的准确率约为35-40%，人类领域专家基准线约为65%，随机猜测为25%68% similarUnverified在HealthBench Professional基准上Mythos 5得66.0%，GPT-5.6 Sol得60.5%（长度校正后约64%）68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/589619API
curl https://kongchang.com/api/v1/knowledge/claims/589619MCP
get_claim(id=589619)