Terminal-Bench 4.0
A programming capability benchmark focused on real-world engineering tasks, emphasizing the evaluation of multi-step command-line task completion in terminal environments. It measures the practical usability of coding agents operating in authentic terminal-based workflows.
Core Facts
On Terminal-Bench 4.0, Claude Opus 5.5 leads with 66.4%, followed by GPT-6 Astra at 61.8% and Argon at 57.4%.
Claude Sonnet 5.5 achieved 70.6% on the Terminal benchmark, compared to only 10.3% for the previous generation.
In the Automation Bench office workflow test, Gemini 4 Argon scored 51.3%, GPT-6 Astra scored 41.4%, and Claude Opus 5.5 scored 42.5%.
Timeline (last 90 days)
The new model improved on all 11 of the vendor's own benchmarks, with Terminal-Bench 4.0 up 11.1 points and ProgramBench up 8.8 points.
According to official self-reported data, compared with V4.1-Flash, V4.1-Flash-1017 improved on Terminal-Bench 4.0 from 31.2 to 42.3 (+11.1), on HLE from 36.8 to 42.6 (+5.8), on DeepSWE v1.1 from 74.2 to 77.0 (+2.8), and on Automation-Bench from 54.8 to 60.5 (+5.7).
DeepSeek-V4.1-Flash-1017's score on Terminal-Bench 4.0 rose from 31.2 for V4.1-Flash to 42.3.
DeepSeek V4.1-Flash-1017 improves by 11 points on the Terminal-Bench 4.0 benchmark compared with the previous version.
Opus leads on Terminal-Bench 4.0, Humanity's Last Exam and long-context reasoning.
Terminal-Bench 4.0 is an evaluation suite for command-line agents, requiring models to complete multi-step operations in real terminal environments, testing coherence and self-correction ability in long-horizon execution.
On Terminal-Bench 4.0, Haiku 5.5 scores 39.2% versus Opus 5.5's 70.6%.
Terminal-Bench 4.0 has a standard error of ±2.6 points, and Opus 5's 51.8% reproduced as 52.3%, which Anthropic calls noise.
Wonder Full trails Opus 5.5 on ProgramBench (59.0 vs 70.5), Terminal-Bench 4.0, and HealthBench Pro.
Argon leads on DeepSWE v1.1 with 77.9% versus Opus 5.5's 74.2%; on Terminal-Bench 4.0, Opus 5.5 leads with 66.4% versus 57.4%.
10 more timeline events
All Facts (20)
On Terminal-Bench 4.0, Claude Opus 5.5 leads with 66.4%, followed by GPT-6 Astra at 61.8% and Argon at 57.4%.
75%VerifiedClaude Sonnet 5.5 achieved 70.6% on the Terminal benchmark, compared to only 10.3% for the previous generation.
75%VerifiedIn the Automation Bench office workflow test, Gemini 4 Argon scored 51.3%, GPT-6 Astra scored 41.4%, and Claude Opus 5.5 scored 42.5%.
65%VerifiedTerminal Bench requires models to complete a full software engineering workflow in a simulated terminal environment, including reading error logs, debugging, configuring environment variables, and operating the file system.
65%UnverifiedThe new model improved on all 11 of the vendor's own benchmarks, with Terminal-Bench 4.0 up 11.1 points and ProgramBench up 8.8 points.
50%UnverifiedAccording to official self-reported data, compared with V4.1-Flash, V4.1-Flash-1017 improved on Terminal-Bench 4.0 from 31.2 to 42.3 (+11.1), on HLE from 36.8 to 42.6 (+5.8), on DeepSWE v1.1 from 74.2 to 77.0 (+2.8), and on Automation-Bench from 54.8 to 60.5 (+5.7).
50%UnverifiedDeepSeek-V4.1-Flash-1017's score on Terminal-Bench 4.0 rose from 31.2 for V4.1-Flash to 42.3.
50%UnverifiedDeepSeek V4.1-Flash-1017 improves by 11 points on the Terminal-Bench 4.0 benchmark compared with the previous version.
50%UnverifiedOpus leads on Terminal-Bench 4.0, Humanity's Last Exam and long-context reasoning.
50%UnverifiedTerminal-Bench 4.0 is an evaluation suite for command-line agents, requiring models to complete multi-step operations in real terminal environments, testing coherence and self-correction ability in long-horizon execution.
50%UnverifiedOn Terminal-Bench 4.0, Haiku 5.5 scores 39.2% versus Opus 5.5's 70.6%.
50%UnverifiedTerminal-Bench 4.0 has a standard error of ±2.6 points, and Opus 5's 51.8% reproduced as 52.3%, which Anthropic calls noise.
50%UnverifiedWonder Full trails Opus 5.5 on ProgramBench (59.0 vs 70.5), Terminal-Bench 4.0, and HealthBench Pro.
50%UnverifiedArgon leads on DeepSWE v1.1 with 77.9% versus Opus 5.5's 74.2%; on Terminal-Bench 4.0, Opus 5.5 leads with 66.4% versus 57.4%.
50%UnverifiedOn the Terminal-Bench 4.0 benchmark, Mistral Large 3 achieved a score of 65.2%, ranking second worldwide.
50%UnverifiedClaude Haiku jumped from 0.68 in version 4.5 to 0.95 in version 5.5, an improvement of 27 points.
50%UnverifiedTerminal-Bench version 4.0 adds multi-step dependency tasks and failure recovery scenarios compared to previous versions.
50%UnverifiedGPT-6 Astra has set new records on the Terminal-Bench 4.0 and OSWorld 2.0 benchmarks.
50%UnverifiedThe vendor officially claims that Weave Router 2.0 performs on par with GPT-6 Astra on the Terminal-Bench 4.0 and SWE-Atlas benchmarks, at half the cost and twice the speed.
50%UnverifiedDeepSeek的编程智能体DeepSWE在SWE-bench基准测试中从预览版的12.8分提升至62.7分
50%