Unverified50% confidenceBenchmarkExact time
TerminalBench 2.0 contains 89 tasks, each run five times; the same model (such as GPT-5.3 Codex or Claude Opus 4.6) shows significant score differences under different harnesses, but the median fluctuation from repeated runs of the same item is about 4.5 percentage points.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/4/2026
First Seen
Valid until: 2/2/2027
Sources
AI Agent Harness Explained: How 66 Frameworks Are Reshaping Agent Capabilities
youtubeAI Briefing Daily11/3/2026
Related Entities
Related Claims
UnverifiedOn the Terminal-Bench 2.0 benchmark, the same model Opus 4.6 scores from 60.45% to 82.02% under different harnesses (a gap of about 22 percentage points).73% similarUnverifiedIn one study on the SWE-bench benchmark, GPT-4 Turbo solved about 1% of problems in a single attempt when given only a few relevant files; the same model with tools and a loop reached a 12.5% solve rate.70% similarUnverifiedIn the Automation Bench test, GPT-6 Sol's extra high tier achieved first place 33% of the time, while 5.6's max effort scored 28.8.68% similarUnverifiedClaude Opus 4.6 produced 30 times more errors than GPT-5.4, averaging 2.5 errors per trajectory vs. 0.08, with the difference being statistically significant (p < 0.0001, Cliff's δ = 0.613).66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1135173API
curl https://kongchang.com/api/v1/knowledge/claims/1135173MCP
get_claim(id=1135173)