Automation Bench
A benchmark that measures the ability of AI models to autonomously complete multi-step tasks in realistic business workflow scenarios. It covers practical tasks such as table parsing, form filling, and workflow triggering.
Core Facts
In the Automation Bench office workflow test, Gemini 4 Argon scored 51.3%, GPT-6 Astra scored 41.4%, and Claude Opus 5.5 scored 42.5%.
Automation Bench is a benchmark that evaluates AI's actual performance in enterprise-level automation scenarios, with end-to-end task completion rate as the core metric and token consumption recorded.
The per-task cost of GPT-6 Sol High on Automation Bench is only one-tenth that of Opus 5.
Timeline (last 90 days)
Argon achieved the best results on the VALS index, Automation Bench, VALS Finance Agent and Harvey legal Agent benchmarks.
WonderFull reaches state-of-the-art levels on the Harvey Lab legal work benchmark, Val's financial agent, and Automation Bench.
On the Automation Bench leaderboard, Claude Opus 5's per-task cost is 11 times that of Sol.
On the Automation Bench benchmark, Sol achieved a score of 33.2% with a per-task cost of $0.27.
In the Automation Bench office workflow test, Gemini 4 Argon scored 51.3%, GPT-6 Astra scored 41.4%, and Claude Opus 5.5 scored 42.5%.
On Automation Bench, Gemini 4 Argon scored 51.3%, surpassing Fable 5.1's 31.4% and Opus 5.5's 42.5%.
At the highest-tier setting on Automation Bench, Opus 5.5 achieves a task success rate of 40%, with an average model cost of approximately $1.28 per attempt.
Automation Bench uses whether a task is ultimately completed in full as the core evaluation criterion, rather than the correctness rate of intermediate steps.
On the Automation Bench test, the high-tier Sol scored higher than the highest-tier Cloud Ops 5, with a per-question cost of only 9% of its competitor's.
In Automation Bench at medium reasoning intensity, GPT-6.1 Sol scored 35.4%, which is 2.2 percentage points higher than Opus 5.5 at one-third of the cost.
10 more timeline events
All Facts (18)
In the Automation Bench office workflow test, Gemini 4 Argon scored 51.3%, GPT-6 Astra scored 41.4%, and Claude Opus 5.5 scored 42.5%.
65%VerifiedAutomation Bench is a benchmark that evaluates AI's actual performance in enterprise-level automation scenarios, with end-to-end task completion rate as the core metric and token consumption recorded.
65%Partially VerifiedThe per-task cost of GPT-6 Sol High on Automation Bench is only one-tenth that of Opus 5.
65%UnverifiedIn the Automation Bench test, GPT-6 Sol's extra high tier achieved first place 33% of the time, while 5.6's max effort scored 28.8.
60%UnverifiedDeepSeek's model V4 Flash 0731 achieved a score of 82.7% on Terminal-Bench 2.1.
60%UnverifiedArgon achieved the best results on the VALS index, Automation Bench, VALS Finance Agent and Harvey legal Agent benchmarks.
50%UnverifiedWonderFull reaches state-of-the-art levels on the Harvey Lab legal work benchmark, Val's financial agent, and Automation Bench.
50%UnverifiedOn the Automation Bench leaderboard, Claude Opus 5's per-task cost is 11 times that of Sol.
50%UnverifiedOn the Automation Bench benchmark, Sol achieved a score of 33.2% with a per-task cost of $0.27.
50%UnverifiedAutomation Bench uses whether a task is ultimately completed in full as the core evaluation criterion, rather than the correctness rate of intermediate steps.
50%UnverifiedAt the highest-tier setting on Automation Bench, Opus 5.5 achieves a task success rate of 40%, with an average model cost of approximately $1.28 per attempt.
50%UnverifiedOn the Automation Bench test, the high-tier Sol scored higher than the highest-tier Cloud Ops 5, with a per-question cost of only 9% of its competitor's.
50%UnverifiedIn Automation Bench at medium reasoning intensity, GPT-6.1 Sol scored 35.4%, which is 2.2 percentage points higher than Opus 5.5 at one-third of the cost.
50%UnverifiedIn the Automation Bench test, the score improved from Cell's 8.1 to Astra's 41.4, and the CAD task score improved from 83.3 to 95.9.
50%UnverifiedThe release party claimed that DeepSeek-V4.1-Flash outperforms Opus 5 and GPT-5.6 Sol on three benchmarks: DeepSWE, CyberGym, and Automation Bench.
50%UnverifiedDeepSWE, CyberGym, and Automation Bench correspond respectively to evaluations for software engineering, cybersecurity, and automated agent tasks.
50%UnverifiedOn Automation Bench, the old model scored 17%, while Gemini 3.7 Flash reached 30.4%.
50%Unverified在Automation Bench上Claude Opus 5拿到约25%通过率，而其他模型普遍聚集在15%左右
50%