[KongchangAI]
BenchmarkTerminal Bench 4.1

Terminal Bench 4.1

A benchmark that evaluates the ability of large language models to perform tasks in terminal (command-line) environments, covering scenarios such as file operations, script writing and system command invocation.

Core Facts

Timeline (last 90 days)

Nov 15

Different versions of the same benchmark (such as Terminal Bench 4.0 and 4.1) have different problem sets and scoring rules, and scores cannot be compared across versions.

Unverified50%
Nov 15

On the same Terminal Bench, Claude Sonnet 5.5 scores 63.9, ahead of Hydrogen's 57%; on computer use tasks Sonnet wins 83.9 to 77.6.

Unverified60%
Nov 14

On Terminal Bench 4.1, Gemini 4 Hydrogen reduces fabricated statements by more than 5 times and handles failed tool calls more stably.

Unverified50%
Nov 14

On Terminal Bench 4.1, Hydrogen's score rose from 18.2% to 57%, and the proportion of fabricated assertions dropped below 1.5%.

Unverified50%
Nov 14

Gemini 4 Hydrogen scored 57% on the Terminal Bench 4.1 test, more than three times the previous generation.

Unverified50%
Nov 12

Migrating from Gemini 3.8 Flash to Hydrogen, at the default high setting, task completion speed increases 2.5 times, the Terminal-Bench 4.1 score rises from 18.2% to 57%, and fabricated claims decrease by more than 5 times.

Verified90%
Oct 14

Terminal-Bench 4.0 is an evaluation suite for command-line agents, requiring models to complete multi-step operations in real terminal environments, testing coherence and self-correction ability in long-horizon execution.

Verified65%

All Facts (7)

Source Articles