GPT-5.6 Three-Model Benchmark: How Close Are Sol/Terra/Luna to Claude?

OpenAI GPT-5.6 Series First Look: Full Evaluation of Sol, Terra, and Luna

OpenAI has begun rolling out preview access to the GPT-5.6 series, releasing three models at once. As a content creator who regularly runs model benchmarks, I got early access to all three and put them through my own test suite. This article covers what these models are, how they performed, and my overall take.

Three Models: Sol, Terra, and Luna

OpenAI released the GPT-5.6 series on June 26, currently available only via API in limited preview, with access also extended to select partners through Codex. A general rollout is expected within weeks.

You may not have noticed that before the public launch, OpenAI demonstrated these models to the U.S. government at the government's request, using a phased release approach — a first in the company's history. This cautious rollout strategy itself signals that the models have reached a significant capability threshold, particularly in cybersecurity. Proactively briefing regulators before release isn't standard practice, and the likely driver is strong performance on security-sensitive benchmarks like Exploit Bench: when a general-purpose LLM develops meaningful vulnerability analysis and exploitation capabilities, front-loading regulatory review becomes a necessary risk control measure. Exploit Bench specifically measures AI capabilities in offensive and defensive cybersecurity — covering vulnerability identification, exploit chain construction, and penetration testing scenarios — and belongs to the "task completion" evaluation paradigm. Unlike traditional multiple-choice or text generation scoring, this type of evaluation requires models to complete end-to-end penetration testing workflows in sandboxed environments, more directly reflecting a model's practical threat level in real-world attack-defense scenarios, which is precisely the dimension regulators care most about.

On naming, OpenAI explains that the number represents the "generation," while Sol, Terra, and Luna are "capability tiers" that will be retained long-term and iterated at their own cadence. This naming strategy marks an important shift for OpenAI — from version-number iteration to capability-tier branding — closely mirroring Anthropic's approach of dividing the Claude series into Haiku, Sonnet, and Opus tiers. Fixed tier names help developers build lasting product intuitions without having to re-evaluate their choices every update. For enterprise customers, a stable naming system lowers migration decision costs; for OpenAI, it allows flexible updates to underlying models without frequently changing API endpoint names:

• Sol: Flagship, most capable
• Terra: Balanced model for everyday work
• Luna: Fast and affordable

This naming system will very likely stick around for the long haul.

Official Capabilities and Pricing

From the officially published data, GPT-5.6 performs strongly across multiple benchmarks. OpenAI claims it achieves state-of-the-art results on Terminal Bench 2.1.

Terminal Bench is a benchmark specifically designed to evaluate AI performance on complex tasks in command-line environments — fundamentally different from traditional conversational evaluations. It requires models to autonomously complete multi-step tasks in real or simulated shell environments, covering file system operations, process management, scripting, and dependency debugging, testing multi-step planning, tool invocation, and iterative error correction. Version 2.1 adds a multi-tool coordination dimension (e.g., simultaneously operating git, docker, and package managers), more closely reflecting real DevOps and systems engineering scenarios. This "task completion" evaluation paradigm reflects more practical engineering value than traditional multiple-choice or text scoring, and has become a core metric for measuring generational leaps in "agentic capability."

Results on long-horizon biology task benchmarks are also strong, and on Exploit Bench, Sol holds its own against Anthropic's models while using roughly one-third the output tokens. This explains why OpenAI chose to brief the government before public release — when a model makes a substantive breakthrough in security-sensitive capabilities, a cautious phased rollout becomes a necessary risk management measure.

This release also introduces the highest reasoning intensity tier yet, along with a new Ultra mode. Ultra mode uses a multi-agent concurrent architecture: the system decomposes complex tasks and simultaneously dispatches multiple sub-agents to handle different subtasks in parallel, with a coordination layer aggregating the results. Architecturally, this mode fuses two design paradigms — "mixture of agents" and "orchestrator-worker." The former emphasizes ensemble voting across multiple homogeneous models to improve result robustness; the latter emphasizes distributing heterogeneous subtasks to specialized sub-agents for greater depth. Ultra mode can significantly improve output quality for tasks requiring parallel reasoning or multi-perspective validation, but correspondingly multiplies token consumption. It's best suited for high-value use cases where quality demands are extreme and cost sensitivity is low — such as code audits, complex research report generation, or critical business decision support.

Pricing (per million tokens)

Model	Input	Output
Sol	$5	$30
Terra	$2.5	Competitive
Luna	$1	$6

OpenAI says Terra is priced at half the cost of GPT-5.5. They've also improved prompt caching — a technique that temporarily stores repeated context prefixes server-side to avoid redundant computation. When a model receives input sharing the same prefix as a prior request, it reuses the already-computed KV cache (key-value cache), skipping attention computation for that prefix, thereby reducing both latency and billing costs. GPT-5.6 introduces explicit cache breakpoints, allowing developers to precisely control what enters the cache — more predictable than the implicit caching of previous versions. This improvement is especially valuable for agentic applications: in typical multi-turn agent workflows, system prompts and tool definitions often occupy the majority of the context window; recalculating them on every call causes costs to grow linearly with the length of the call chain. Explicit cache control lets developers factor cost into system architecture design rather than optimizing reactively. Cache retention is at least 30 minutes, and output speeds can reach hundreds of tokens per second. From a cost and speed perspective, this pricing is genuinely competitive.

Benchmark Testing: King Bench 3 Results

I ran all three models through my King Bench 3 test suite, covering frontend tasks, Three.js tasks, SVG generation, a high-difficulty math problem, and a long-horizon agentic task.

A long-horizon agentic task requires a model to autonomously complete complex workflows with multiple dependent steps — without human intervention. This is fundamentally different from single-turn Q&A; it requires models to track state across steps, recover from intermediate errors, and dynamically adjust subsequent action plans. These tasks are a core metric for measuring how far a model has evolved from "intelligent assistant" to "autonomous execution unit," with far-reaching implications for MLOps, DevOps, and low-code platforms. Specifically, the task here required the model to generate a dataset, fine-tune a model locally, and then build a Web UI around the entire pipeline — a closed loop that previously required a dedicated MLOps toolchain.

Overall Scores (out of 70):

• Sol: 55 points (~78.6%)
• Terra: ~62.9%
• Luna: 31 points (~44.3%)

The Subcategories Tell the Real Story

Don't let Sol's total score discourage you — the individual category results are far more revealing than the aggregate.

On the high-difficulty math problem, both Sol and Terra scored a perfect 10/10 — very few models can actually solve this question.

Even more impressive was the long-horizon agentic fine-tuning task: all three GPT-5.6 models scored a perfect 10/10, including the cheapest Luna, which successfully completed the full pipeline of dataset generation, model fine-tuning, and web UI creation. This result carries an important industry signal: when a general-purpose model can autonomously complete a full closed-loop workflow that previously required a professional MLOps toolchain — from data labeling and training script writing to service deployment — automation in software development and data engineering will see a substantial leap. Workflows that once required multiple specialized roles collaborating could be compressed into a single agent node. For comparison, GPT-5.5 and Opus 4.7 could only score two or three points on this same task. On long-horizon backend agentic tasks, these models are genuinely impressive.

Frontend Is the Weak Spot

Their clear weakness is in frontend tasks. For visually oriented projects like Three.js work, folding table animations, and SVG pandas, Sol generally scores around six or seven points, while Fable 5 can score nine or even a perfect ten on the same items.

To be honest, my test suite is front-end and visually heavy — because that's where quality differences are immediately visible. Three.js and other 3D rendering frameworks require models to simultaneously master WebGL shader syntax, scene graph management, and precise lighting model implementation; any coordinate transformation or matrix math error directly causes visible rendering artifacts. SVG generation requires precise geometric intuition around Bézier curve path instructions, viewBox coordinate systems, and CSS transformation matrices. This type of "visual-code alignment" ability — mapping semantic descriptions to pixel-accurate rendering output — has historically been the weakest area for text-only training paradigms, requiring large quantities of high-quality "code-render result" paired training data, and is the dimension most significantly impacted by multimodal input. So GPT-5.6's scores in my benchmark are somewhat underestimated — it's not that the models are weak, but frontend remains one of OpenAI's relative weaknesses, and my benchmark happens to be particularly demanding in this area.

Cross-Model Comparison: vs. Fable 5 and Opus

Here's how the models rank on my overall leaderboard:

1. Fable 5: 88.57% (1st)
2. Opus 4.8: 87.14%
3. GM 5.2: 81.43%
4. Sol: 78.57%
5. GPT-5.5: 38.57%

Sol's score is roughly double that of the previous-generation GPT-5.5 — a substantial improvement. This generational leap in capability — particularly in agentic tasks and mathematical reasoning — likely reflects a fundamental shift in training paradigm: moving from pure instruction fine-tuning toward reinforcement learning combined with a process reward model (PRM). PRM doesn't just reward final answers during training; it also provides fine-grained feedback on the quality of intermediate reasoning steps. This gives the model qualitative gains in scenarios requiring multi-step reasoning, self-verification, and intermediate error correction, rather than just "getting lucky" at the final output stage — which aligns closely with GPT-5.6's strong performance on long-horizon agentic tasks.

Overall Assessment and Model Selection Guide

This is a solid incremental upgrade, not a paradigm shift. The improvements are real and meaningful, but fundamentally this is a natural next step from GPT-5.5 — just stronger and cheaper. The Sol/Terra/Luna naming feels somewhat marketing-driven — OpenAI seems to be taking a page from Anthropic's playbook of packaging Fable as a "new category of model," making their own release feel like a full new product line.

Has it reached Fable's level? Not quite. Fable remains the strongest overall performer in my testing, visible across nearly every category. That said, Sol is very close, and even surpasses Fable in some tasks — particularly in cybersecurity performance and token efficiency on agentic tasks. On long-horizon agentic tasks specifically, it's a dead heat with Fable.

Fable's clear weakness is price — it's the most expensive option available. Luna, meanwhile, is among the cheapest in its capability tier. For applications requiring high-frequency calls, $1/million input tokens combined with improved prompt caching can keep actual operating costs remarkably low. In multi-turn agentic scenarios with long system prompts, the higher the cache hit rate, the more the marginal cost per call approaches zero — if system prompts account for more than 60% of total tokens, the actual billed cost after cache hits may be only 30–40% of the list price. This means monthly API bills for high-concurrency agentic applications could be more than half what intuition would suggest. For teams deploying agentic workflows at scale, this is economically significant.

Model Selection Guide

• Heavy frontend/visual tasks: Go with Fable 5 or Opus 4.8
• Backend agentic or long-horizon tasks: Sol is a strong choice at much lower cost
• Cost-efficient agentic scenarios: Luna ($1/million input tokens) is probably the best value right now

Note: The above conclusions are based on preview version testing. Data may change before the official release and is provided for reference only.

Closing Thoughts

This release is more understated than the marketing suggests — it's not a paradigm shift, but it is a solid, reasonably priced upgrade that closes most of the gap with Fable. Overall, the GPT-5.6 series is a lineup worth taking seriously. Feel free to share your thoughts in the comments.