Terminal Bench
A benchmark designed to evaluate AI performance on complex tasks within command-line environments. It covers scenarios including file system operations, process management, script writing, and dependency debugging. Version 2.1 introduced a multi-tool coordination dimension.
Core Facts
Terminal-Bench is a benchmark specifically designed to evaluate AI models' ability to execute complex tasks in command-line environments, covering scenarios such as file operations, script writing, and system debugging.
Claude Sonnet 5.5 achieved 70.6% on the Terminal benchmark, compared to only 10.3% for the previous generation.
On the SWE Bench Pro programming benchmark, Sonnet 5 achieved 63.2% and Opus 4.8 achieved 69.2%.
Timeline (last 90 days)
On Terminal Bench, the two Muse models scored 57.1 and 52 respectively, roughly on par with the 52 to 59 range of Fable, Astra, and Opus.
The new model is the first in its series capable of adjusting effort levels, achieving approximately 20% on Terminal Bench at medium effort and only reaching 39.2% at maximum effort.
The new model achieved 1,620 points on GDPVal (compared to 735 for the previous generation), improved on OSWorld from 15.7% to 72.4%, on Terminal Bench from 0.0% to 39.2%, and on Humanity's Last Exam from 10.2% to 45.9% without tools and from 18.7% to 57.4% with tools.
Mistral Large 4 achieved 61.7% on SWE Atlas Q&A, 28.3% on Terminal Bench, and 49.8% on the综合 Coding Agent Index.
In the creator's benchmark, Haiku 5.5's Terminal Bench score improved from 0% on Haiku 4.5 to nearly 40%.
On Terminal Bench, Beam scored approximately 81 points, ranking last, below DeepSeek's 90.6, Kimi's 88.3, GLM's 88.2, and Qwen's 86.6.
In Terminal Bench testing, GPT-6.1 Soul's max/high performance is nearly on par with Opus 5.5, but the cost per task is approximately $0.96 cents, while Opus costs $0.11 to $0.13 per task, roughly one-thirteenth of the price.
In 4 agentic coding tests, Argon won two, and ranked last in both the Frontier SWE and Terminal Bench tests.
Claude Opus 5.5 scores approximately 66.4 on Terminal Bench.
On Terminal Bench, mid-tier Sonnet 5.5 scores 28.8% at approximately $0.83 per task, while Sonnet 5's highest tier only reaches 10.3% at approximately $11.6 per task
29 more timeline events
All Facts (20)
Terminal-Bench is a benchmark specifically designed to evaluate AI models' ability to execute complex tasks in command-line environments, covering scenarios such as file operations, script writing, and system debugging.
90%VerifiedClaude Sonnet 5.5 achieved 70.6% on the Terminal benchmark, compared to only 10.3% for the previous generation.
75%VerifiedOn the SWE Bench Pro programming benchmark, Sonnet 5 achieved 63.2% and Opus 4.8 achieved 69.2%.
70%VerifiedOn Terminal Bench 4.0, Opus 5.5 scored 66.4%, Astra scored 57.9%, Fable 5.1 scored 55.8%, and the previous generation Opus 5 scored 52.3%.
65%VerifiedTerminal Bench requires models to complete a full software engineering workflow in a simulated terminal environment, including reading error logs, debugging, configuring environment variables, and operating the file system.
65%Partially VerifiedThe per-task cost of GPT-6 Sol High on Automation Bench is only one-tenth that of Opus 5.
65%UnverifiedOn Terminal Bench, mid-tier Sonnet 5.5 scores 28.8% at approximately $0.83 per task, while Sonnet 5's highest tier only reaches 10.3% at approximately $11.6 per task
60%UnverifiedFable 5.1 scored 52.6% on the Terminal Bench Science 0.1 benchmark, more than doubling the score of its previous generation.
60%UnverifiedQwen3.8-27B trails Claude Opus 4.6 on Terminal Bench, GPQA Diamond, and HLE benchmarks (73 vs 78.2, 89.2 vs 91.3, and 30.8 vs 40, respectively).
60%UnverifiedDeepSeek officially announced that V4 Pro scored as high as 87.9 on the Terminal Bench benchmark.
60%UnverifiedAI evaluation is shifting from pure language capability to real-world task execution capability, particularly in quantifiable office and engineering scenarios such as terminal operations and spreadsheet analysis.
60%UnverifiedTerminal Bench is an agent evaluation framework designed by Stanford University that simulates a real Linux terminal environment, requiring agents to complete full workflows including environment configuration, dependency installation, code debugging, and result verification.
60%UnverifiedOn Terminal Bench, the two Muse models scored 57.1 and 52 respectively, roughly on par with the 52 to 59 range of Fable, Astra, and Opus.
50%UnverifiedThe new model achieved 1,620 points on GDPVal (compared to 735 for the previous generation), improved on OSWorld from 15.7% to 72.4%, on Terminal Bench from 0.0% to 39.2%, and on Humanity's Last Exam from 10.2% to 45.9% without tools and from 18.7% to 57.4% with tools.
50%UnverifiedThe new model is the first in its series capable of adjusting effort levels, achieving approximately 20% on Terminal Bench at medium effort and only reaching 39.2% at maximum effort.
50%UnverifiedMistral Large 4 achieved 61.7% on SWE Atlas Q&A, 28.3% on Terminal Bench, and 49.8% on the综合 Coding Agent Index.
50%UnverifiedIn the creator's benchmark, Haiku 5.5's Terminal Bench score improved from 0% on Haiku 4.5 to nearly 40%.
50%UnverifiedOn Terminal Bench, Beam scored approximately 81 points, ranking last, below DeepSeek's 90.6, Kimi's 88.3, GLM's 88.2, and Qwen's 86.6.
50%UnverifiedIn Terminal Bench testing, GPT-6.1 Soul's max/high performance is nearly on par with Opus 5.5, but the cost per task is approximately $0.96 cents, while Opus costs $0.11 to $0.13 per task, roughly one-thirteenth of the price.
50%UnverifiedIn 4 agentic coding tests, Argon won two, and ranked last in both the Frontier SWE and Terminal Bench tests.
50%