[KongchangAI]
Benchmark

Frontier Code

A third-party programming benchmark maintained by Cognition, the developer of Devin AI. It is designed to measure the capabilities of long-horizon coding agents and is considered closer to real-world production scenarios than HumanEval.

Core Facts

Timeline (last 90 days)

Nov 21

On the Terminal Bench and Frontier Code benchmarks, Claude Opus 5.5 performed slightly better than Gemini 4 Neon.

Unverified50%
Nov 9

Fable 5.2 scored 49.1% on the Frontier Code coding benchmark and 69.9% on the Frontier SWE agentic coding benchmark, ranking first on both.

Unverified50%
Nov 9

Claude Fable 5.2 scored 49.1% on the FrontierCode 2.0 benchmark, ranking first.

Verified70%
Oct 4

On the Frontier Code reasoning benchmark, the medium reasoning tier performed better than the high and extra high tiers, approaching the max tier; this phenomenon occurred with both Opus and Fable.

Unverified50%
Oct 4

On long-horizon programming tasks such as the Frontier Code benchmark, Opus 5.5 may slightly outperform Astra

Expired50%
Oct 2

Excessive reasoning can hurt performance, with x-high scoring higher than Max on the Frontier Code benchmark.

Unverified50%
Sep 30

Sonnet 5.5's scores on Frontier Code-related benchmarks have risen from 42 to 46 and 52.

Unverified50%
Sep 25

On Frontier Code, Opus 5.5's medium tier (under $1 per task) actually scored higher than its max tier (over $5 per task).

Unverified50%
Sep 25

On Frontier Code, the performance gap among GPT-6 Sol, Luna, and Opus 5.5 is only about 6–7 percentage points, but their prices range from 10 cents, to $2, to $4.

Unverified50%
Sep 25

In the Frontier Code test, GPT-6 Luna Max achieved 66.6%, with a cost of only 22 cents per task.

Unverified50%

4 more timeline events

All Facts (14)

Verified

FrontierCode 1.1 is an evaluation benchmark for AI programming capabilities that simulates real software engineering tasks, including scenarios such as code completion, bug fixing, multi-file refactoring, and unit test generation.

75%
Verified

Claude Fable 5.2 scored 49.1% on the FrontierCode 2.0 benchmark, ranking first.

70%
Verified

Claude Opus 5 approaches Fable-level performance on the FrontierCode 1.1 benchmark while costing only half as much

70%
Unverified

On the Terminal Bench and Frontier Code benchmarks, Claude Opus 5.5 performed slightly better than Gemini 4 Neon.

50%
Unverified

Fable 5.2 scored 49.1% on the Frontier Code coding benchmark and 69.9% on the Frontier SWE agentic coding benchmark, ranking first on both.

50%
Unverified

On the Frontier Code reasoning benchmark, the medium reasoning tier performed better than the high and extra high tiers, approaching the max tier; this phenomenon occurred with both Opus and Fable.

50%
Unverified

Excessive reasoning can hurt performance, with x-high scoring higher than Max on the Frontier Code benchmark.

50%
Unverified

Sonnet 5.5's scores on Frontier Code-related benchmarks have risen from 42 to 46 and 52.

50%
Unverified

On Frontier Code, Opus 5.5's medium tier (under $1 per task) actually scored higher than its max tier (over $5 per task).

50%
Unverified

On Frontier Code, the performance gap among GPT-6 Sol, Luna, and Opus 5.5 is only about 6–7 percentage points, but their prices range from 10 cents, to $2, to $4.

50%
Unverified

In the Frontier Code test, GPT-6 Luna Max achieved 66.6%, with a cost of only 22 cents per task.

50%
Unverified

Cognition has released coding model Sway 2, achieving 50.0% on the Frontier Code benchmark, claiming to be within 1 point of stronger models while costing 64% less.

50%
Unverified

On the Frontier Code programming benchmark, the old model scored 34.4%, while Gemini 3.7 Flash improved to 43.6%.

50%
Unverified

In Cognition's Frontier Code evaluation, Fable 5 ranked first among all frontier models even with only medium reasoning intensity enabled.

50%

Source Articles