TerminalBench 2.0
An evaluation benchmark targeting the capabilities of AI agents in terminal and command-line environments, comprising multiple tasks run repeatedly to assess agent performance under different harness configurations.
Timeline (last 90 days)
TerminalBench 2.0 contains 89 tasks, each run five times; the same model (such as GPT-5.3 Codex or Claude Opus 4.6) shows significant score differences under different harnesses, but the median fluctuation from repeated runs of the same item is about 4.5 percentage points.
The 66-harness map built by the video's author (as of November 1, 2026) has inclusion criteria of: the system's documentation presents it as an agent that acts in a loop on a computer, and it appears in at least one of the TerminalBench 2.0 leaderboard, the six systems classified in the preprint, the Paperclip and OpenHands support lists, or the largest platform-native agents; another nine were included under weaker criteria.
All Facts (2)
TerminalBench 2.0 contains 89 tasks, each run five times; the same model (such as GPT-5.3 Codex or Claude Opus 4.6) shows significant score differences under different harnesses, but the median fluctuation from repeated runs of the same item is about 4.5 percentage points.
50%UnverifiedThe 66-harness map built by the video's author (as of November 1, 2026) has inclusion criteria of: the system's documentation presents it as an agent that acts in a loop on a computer, and it appears in at least one of the TerminalBench 2.0 leaderboard, the six systems classified in the preprint, the Paperclip and OpenHands support lists, or the largest platform-native agents; another nine were included under weaker criteria.
50%