Expired90% confidenceFactTime unknown
After replacing GPT-4 Mini with a stronger answering LLM (Claude Sonnet 4), only the missing task showed a slight improvement, with no notable improvement on cascading tasks.
1
Sources
90%
Confidence
Medium-term (~90 days)
Relevance
6/1/2026
First Seen
Valid until: 8/30/2026(expired)
Sources
Related Entities
Related Claims
UnverifiedIn SWE-bench testing, the early GPT-4 resolution rate was below 2%, while Claude 3.5 Sonnet broke through 49%.77% similarUnverifiedThe performance improvement of Claude 4.5 Sonnet in actual use is only about 10%, far from matching Opus's capabilities on complex tasks such as debugging and planning.73% similarUnverifiedThe gap in resolution rate on SWE-Bench Verified between Claude Sonnet 4.6 and GPT-5.1 Codex is only 3.3 percentage points.71% similarUnverifiedCompared to Opus 4.6, Sonnet 4.6 performs nearly on par on most benchmarks, but with faster inference speed and lower call costs.70% similarUnverifiedFrom Claude 3 Sonnet to Claude 3.5 Sonnet and then to Claude 4 Sonnet, the score improvement on coding benchmarks with each upgrade shows a diminishing trend, with early version-to-version jumps potentially reaching 10–20 percentage points, while the gap between subsequent versions often narrows to 2–5 percentage points.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/21554API
curl https://kongchang.com/api/v1/knowledge/claims/21554MCP
get_claim(id=21554)