Unverified50% confidenceBenchmarkExact time
In one study on the SWE-bench benchmark, GPT-4 Turbo solved about 1% of problems in a single attempt when given only a few relevant files; the same model with tools and a loop reached a 12.5% solve rate.
1
Sources
50%
Confidence
Long-term
Relevance
10/25/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn the Automation Bench test, GPT-6 Sol's extra high tier achieved first place 33% of the time, while 5.6's max effort scored 28.8.71% similarUnverifiedThe same parameter name reasoning_effort behaves differently across models from different vendors: in gpt-5.4-mini the median output tokens rose from 18 at none to 254 at high (×14.1); gpt-oss-120b rose from refusal to 1,340 (×4.6); claude-haiku-5.5 rose from 150 to 298 (×2.0); gemini-3.7-flash rose from 453 to 1,013 (×2.2).70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1096062API
curl https://kongchang.com/api/v1/knowledge/claims/1096062MCP
get_claim(id=1096062)