Frontier Code
A third-party programming benchmark maintained by Cognition, the developer of Devin AI. It is designed to measure the capabilities of long-horizon coding agents and is considered closer to real-world production scenarios than HumanEval.
Timeline (last 90 days)
On the Frontier Code reasoning benchmark, the medium reasoning tier performed better than the high and extra high tiers, approaching the max tier; this phenomenon occurred with both Opus and Fable.
On long-horizon programming tasks such as the Frontier Code benchmark, Opus 5.5 may slightly outperform Astra
Excessive reasoning can hurt performance, with x-high scoring higher than Max on the Frontier Code benchmark.
Sonnet 5.5's scores on Frontier Code-related benchmarks have risen from 42 to 46 and 52.
On Frontier Code, Opus 5.5's medium tier (under $1 per task) actually scored higher than its max tier (over $5 per task).
On Frontier Code, the performance gap among GPT-6 Sol, Luna, and Opus 5.5 is only about 6–7 percentage points, but their prices range from 10 cents, to $2, to $4.
In the Frontier Code test, GPT-6 Luna Max achieved 66.6%, with a cost of only 22 cents per task.
Cognition has released coding model Sway 2, achieving 50.0% on the Frontier Code benchmark, claiming to be within 1 point of stronger models while costing 64% less.
On the Frontier Code programming benchmark, the old model scored 34.4%, while Gemini 3.7 Flash improved to 43.6%.
In Cognition's Frontier Code evaluation, Fable 5 ranked first among all frontier models even with only medium reasoning intensity enabled.
All Facts (10)
On the Frontier Code reasoning benchmark, the medium reasoning tier performed better than the high and extra high tiers, approaching the max tier; this phenomenon occurred with both Opus and Fable.
50%UnverifiedOn long-horizon programming tasks such as the Frontier Code benchmark, Opus 5.5 may slightly outperform Astra
50%UnverifiedExcessive reasoning can hurt performance, with x-high scoring higher than Max on the Frontier Code benchmark.
50%UnverifiedSonnet 5.5's scores on Frontier Code-related benchmarks have risen from 42 to 46 and 52.
50%UnverifiedOn Frontier Code, Opus 5.5's medium tier (under $1 per task) actually scored higher than its max tier (over $5 per task).
50%UnverifiedOn Frontier Code, the performance gap among GPT-6 Sol, Luna, and Opus 5.5 is only about 6–7 percentage points, but their prices range from 10 cents, to $2, to $4.
50%UnverifiedIn the Frontier Code test, GPT-6 Luna Max achieved 66.6%, with a cost of only 22 cents per task.
50%UnverifiedCognition has released coding model Sway 2, achieving 50.0% on the Frontier Code benchmark, claiming to be within 1 point of stronger models while costing 64% less.
50%UnverifiedOn the Frontier Code programming benchmark, the old model scored 34.4%, while Gemini 3.7 Flash improved to 43.6%.
50%UnverifiedIn Cognition's Frontier Code evaluation, Fable 5 ranked first among all frontier models even with only medium reasoning intensity enabled.
50%