GPT-5.6 In-Depth Hands-On Review: How Do Soul, Tara, and Luna Really Perform?

OpenAI's GPT-5.6 Preview Is Here

On June 26, OpenAI quietly rolled out the preview version of the GPT-5.6 series, releasing three distinctly positioned models at once: the flagship Soul, the balanced everyday-office-focused Tara, and the speed-and-value-oriented Luna. Currently, all three models are available through a limited API preview, with Codex access opened to a small number of partners. The company says a full release will arrive in the coming weeks.

One detail worth noting: prior to the official launch, these models were provided to the U.S. government for preview to assess their capabilities, and, at the government's request, adopted a phased rollout strategy—a first in OpenAI's product release history. To some extent, this hints at the capability level the new models have reached in sensitive domains such as cybersecurity.

Industry context: Against the backdrop of increasingly tight AI safety regulation, providing preview evaluations to governments has become the new norm for top-tier AI labs. The U.S. AI Safety Institute (AISI) and its namesake in the UK have, since 2023, begun signing cooperation agreements with major AI companies, requiring "pre-deployment evaluations" of frontier models before public release, with a focus on model performance across "dangerous capability" dimensions such as bioweapon synthesis, cyberattack automation, and nuclear proliferation information. The core methodology of this evaluation framework is the Dangerous Capability Evaluation (DCE): evaluators construct specialized adversarial prompts to test whether a model can provide substantive assistance (i.e., help beyond what publicly available information could offer) in carrying out large-scale harm. OpenAI's proactive cooperation with a phased rollout this time is consistent with Anthropic's "Responsible Scaling Policy (RSP)" and DeepMind's "Frontier Safety Framework"—the RSP requires Anthropic to automatically trigger additional safety measures when a model possesses specific dangerous capabilities, effectively installing a "circuit breaker" for model development. This reflects the entire industry's collective anxiety about safety governance as capability boundaries expand: when model capabilities approach or even surpass human expert levels, technical alignment alone is no longer sufficient, and structural regulatory intervention is becoming an industry consensus.

This article is based on a real-world review from a Bilibili content creator who obtained early beta access, combined with their self-built benchmark KingBench 3, to examine the actual quality of the three GPT-5.6 models and the real gap between them and Anthropic's current strongest model, Fable.

Naming Logic and Pricing Strategy

Numbers Represent Generations, Names Represent Tiers

OpenAI has offered a new explanation for this naming scheme: the number (5.6) represents the model generation, while Soul, Tara, and Luna represent long-term capability tiers that each iterate independently. This naming system will be retained long-term. This shares a certain kinship with Anthropic's positioning of Fable as an "entirely new model category"—both attempt to give users a fresh impression.

The business logic behind the naming system: OpenAI's version-naming strategy reflects the industry trend of large language models evolving from a "single flagship" toward a "model family." The earlier GPT-4 series had already attempted to differentiate tiers through suffixes like GPT-4o and GPT-4o mini, but still relied on a single main version number. GPT-5.6 introduces a dual-track "number + proper noun" system, essentially borrowing the product-line management logic of the chip industry—much like how NVIDIA layers GPU architecture codenames (Hopper, Ada) with SKUs (H100, H800), allowing different-spec products within the same generation to iterate independently rather than being bound to the same release cadence. This strategy also carries a deeper competitive intent: after Anthropic built a three-tier product line with "Sonnet/Opus/Haiku" and Google divided tiers with "Flash/Pro/Ultra," major AI companies have generally realized that a single flagship model creates user inertia around "waiting for the new version." Giving mid- and low-end models independent brand names effectively grants them independent market life cycles, allowing them to maintain user stickiness through gradual performance iteration rather than price wars. From a consumer psychology perspective, "Luna" is easier to establish product recognition than "GPT-5.6 mini," and this named-tier strategy is becoming standard practice among leading AI companies.

Pricing: Quite Generous This Time

The pricing tiers for the three models are clear:

• Soul: $5 input / $30 output (per million tokens)
• Tara: $2.5 input / $15 output—the company claims its capabilities rival GPT-5.5, but at half the price
• Luna: $1 input / $6 output

Technical explanation: LLM pricing is measured in "per million tokens." A token is the basic unit a model uses to process text, roughly corresponding to 0.75 English words or 1.5 Chinese characters—a 1,000-character Chinese article is about 650–700 tokens, and a 500-word English code segment is about 600–700 tokens. Charging separately for input and output stems from the asymmetry of computational cost: generating tokens requires step-by-step autoregressive sampling (each step predicting the next token based on already-generated content), which consumes far more compute than encoding input, so output prices are typically 4–6 times input prices.

The reason behind this pricing structure is the fundamental compute difference between the two phases of "prefill" and "decode": during the prefill phase, when processing input, the attention computation for all input tokens can be highly parallelized, and the GPU's matrix computation units (Tensor Cores) run near peak utilization; during the decode phase, constrained by serial autoregression, only one token can be generated at a time before moving to the next step, leaving many GPU compute units idle, with actual utilization possibly only 10–20% of prefill. This fundamental computational architecture difference makes the marginal cost of output as much as 4–8 times that of input, directly determining the pricing structure.

As knowledge distillation techniques (compressing the implicit knowledge of large models into small-model parameters, rather than simply reducing scale) and speculative decoding (in which a small "draft model" rapidly predicts multiple tokens, which are then verified in parallel by the large model, with rejected tokens rolled back and regenerated) mature, there remains substantial room to improve the cost-effectiveness of mid- and low-end models—this is also the technical foundation of Tara and Luna's long-term commercial competitiveness.

Prompt caching (KV Cache) is a key technique for reducing the cost of repeated requests—in the Transformer attention mechanism, each token needs to compute attention weights against all historical tokens in the sequence, and once its Key and Value matrices are computed, they don't need to be recomputed when processing subsequent tokens and can be read directly from GPU memory (HBM). When the same system prompt is used repeatedly (such as fixed role settings in enterprise deployments), the model can directly reuse the cached KV matrices, entirely skipping the prefill computation, typically reducing latency by 40–60% and cost by 50–90%. OpenAI's introduction of "Explicit Cache Breakpoints" this time means developers can proactively mark which segments of a prompt should be cached preferentially—this is more precise than the previous implicit automatic caching, and delivers especially significant cost-control benefits for enterprise-grade API call scenarios with long system prompts (over 1,000 tokens).

In addition, the new version optimizes the prompt caching mechanism through explicit cache breakpoints, guaranteeing at least 30 minutes of cache validity, with speeds reaching up to 750 tokens per second (about 562 English words per second, suitable for real-time streaming output scenarios). Whether from a cost or speed standpoint, this upgrade is quite pragmatic.

Officially Claimed Performance Highlights

OpenAI claims GPT-5.6 stands out across multiple benchmarks:

• Reaches industry-leading levels on Terminal Bench 2.1 (covering command-line workflow planning, iteration, and tool coordination)
• Performs excellently on the long-horizon task test Gingbench
• On Exploit Bench, Soul's performance rivals Anthropic's Mist Preview while consuming only one-third of the tokens

The new features are equally noteworthy: the model introduces a "maximum reasoning effort" option, as well as a brand-new "super mode" capable of mobilizing agents to handle complex tasks. These capability enhancements are the core value of this upgrade.

Note on benchmark credibility: Terminal Bench 2.1, Gingbench, and Exploit Bench are all practical, real-world benchmarks that have emerged in recent years. Compared with academically common ones like MMLU or MATH, such benchmarks place greater emphasis on a model's execution ability in real engineering environments. Among them, Exploit Bench specifically tests a model's ability to assist in cybersecurity analysis—including vulnerability identification, proof-of-concept code generation, and defense recommendations—making it one of the few standardized tests designed by security red teams that evaluates both offensive and defensive capabilities. Token efficiency (the number of tokens consumed to complete the same task) has been introduced as an additional evaluation dimension, reflecting the industry's shift toward "cost-effective intelligence" rather than high scores alone. It's worth noting that divergence between official benchmarks and third-party real-world benchmarks in rankings often reveals deeper differences in training optimization direction: when a model leads by a wide margin on official benchmarks yet performs mediocrely in community real-world tests, it usually means that the model's reinforcement learning fine-tuning stage was over-optimized for specific benchmark formats (i.e., "benchmark overfitting") rather than comprehensively improving general reasoning ability. GPT-5.6's token efficiency advantage on Exploit Bench, if verified in larger-scale community testing, would be important evidence of its real competitiveness in cybersecurity engineering scenarios.

KingBench 3 Hands-On Test: Subcategory Scores Are More Telling Than Totals

Benchmark Ecosystem Background

The AI benchmark ecosystem is undergoing a paradigm shift from academic standards toward real-world orientation. Traditional benchmarks such as MMLU (Massive Multitask Language Understanding, a collection of multiple-choice questions spanning 57 disciplines) and HumanEval (code completion tests for public GitHub functions) have gradually lost the ability to distinguish top-tier models due to heavy "contamination" from training data—when a model has seen large amounts of data highly similar to the test questions during pretraining, the test score reflects "memorization" rather than "reasoning." This phenomenon is known as "Benchmark Saturation": when GPT-4 was released, its MMLU score was about 86%, whereas today several open-source models have already broken 90%, yet the gap in actual user experience is far less significant than the numerical gap.

Community-built benchmarks have therefore emerged to fill this gap: Chatbot Arena (LMSYS) builds ELO rankings through large-scale blind human testing (users don't know which two models are being compared), offering strong anti-cheating properties; LiveBench uses questions drawn monthly from the latest competitions and papers, fundamentally eliminating training set leakage; and individual-developer benchmarks like KingBench focus more on faithfully reproducing specific workflows—their limitations lie in small sample sizes and strong evaluation subjectivity, but their advantage is that they can quickly respond to trending industry scenarios and capture the "actual usefulness" that official benchmarks struggle to quantify. KingBench 3 covers Three.js 3D rendering, SVG vector graphics generation, mathematical reasoning, and multi-step agentic tasks, making it one of the tests closest to real software engineering scenarios, and a key indicator for distinguishing "models that can chat" from "models that can get work done."

Total Score Performance

The creator used their self-built KingBench 3 for evaluation. The test suite covers front-end development (Three.js development, SVG generation), one high-difficulty math problem, and a long-horizon agentic task—the model must generate a dataset on its own, fine-tune a Gemma model locally, and independently build a complete Web UI.

The total scores of the three models are as follows:

• Soul: 55 points, about 78.6%
• Tara: 44 points, about 62.9%
• Luna: 31 points, about 44.3%

Don't Be Fooled by the Total: Subcategories Are Key

At first glance, Soul's 78.6% seems unremarkable, but the subcategory scores tell a completely different story.

On the high-difficulty math problem, both Soul and Tara scored a perfect 10—an achievement few models can match. And on the agentic fine-tuning task, all three GPT-5.6 models scored perfectly: even the cheapest Luna independently completed the full workflow of dataset generation, Gemma fine-tuning, and web UI construction, scoring on par with Fable 5.

Technical background: An "Agentic Task" fundamentally differs from traditional single-turn Q&A: the model must maintain goal consistency across multiple steps, autonomously invoke tools (file system, code executor, network requests), and self-correct when intermediate steps go wrong. Completing the full workflow of "generate dataset → fine-tune Gemma → build Web UI" requires the model to traverse at least dozens of decision nodes, where an error at any node could cause cascading failure of subsequent steps.

There is a critical "error accumulation effect" here: small deviations in early steps of an agentic task amplify exponentially in later steps, ultimately leading to task failure. In this specific test scenario: if the dataset generation stage has a format error (such as inconsistent field name capitalization), the fine-tuning script may throw a parsing exception; even if fine-tuning succeeds, if hyperparameters are chosen poorly (a learning rate that's too high causing catastrophic forgetting), the final model quality will degrade severely; and if the Web UI stage misunderstands the model API call interface, the entire demonstration workflow will fail to run. Academia models this as a "long-range dependency" problem—the model must maintain a precise understanding of the initial goal across dozens or even hundreds of decision steps and possess "metacognitive" ability (i.e., awareness of whether the current intermediate result deviates from the goal). GPT-5.6's perfect score on such tasks suggests that OpenAI conducted targeted reward design for multi-step tool call chains during the reinforcement learning fine-tuning (RLHF/RLAIF) stage, teaching the model to proactively roll back and try alternative approaches when a tool call fails, rather than blindly continuing with subsequent steps.

Gemma is a series of open-source lightweight language models released by Google DeepMind in February 2024, offered in 2B and 7B parameter sizes. It uses the same technical architecture as Gemini but with drastically reduced scale, designed to run locally on consumer-grade GPUs and even high-end laptops. Due to its small parameter count, low VRAM threshold for fine-tuning (the 7B model can be fine-tuned on a single RTX 4090), and fully open weights that can be deployed locally, Gemma has become an ideal test vehicle for evaluating whether a model "can guide another AI to complete a learning task." Being able to independently complete such a task means the model possesses truly "engineer-level" end-to-end execution capability: it not only understands the task goal but can also dynamically adjust its execution strategy based on the local environment. It's worth mentioning that "Catastrophic Forgetting" is a core challenge when fine-tuning small language models—when a model is continuously trained on task-specific data, it gradually "forgets" the general knowledge acquired during pretraining, resulting in a fine-tuned model that excels on the new task but degrades sharply in basic reasoning. Being able to anticipate and avoid this problem within fine-tuning instructions demonstrates a higher level of engineering judgment.

By comparison, GPT-5.5 and Opus 4.7 often score only two or three points on the same task. In other words, on long-horizon backend agent tasks, the GPT-5.6 series performs exceptionally well.

Front-End Development Remains a Weakness

But on front-end and visual tasks, the models clearly struggle. On the Three.js contact lens case, the folding table animation, and the SVG panda case, Soul could only score six to seven points, while Fable 5 scored nine to ten.

Why front-end is harder: Three.js is a WebGL-based JavaScript 3D graphics library that wraps the underlying OpenGL ES shader language (GLSL) into a JavaScript API, enabling real-time 3D rendering in the browser. SVG (Scalable Vector Graphics) is an XML-based 2D vector graphics standard that describes shapes, paths, and styles through precise mathematical coordinates. Both types of tasks require the model to accurately translate abstract visual intent into code with spatial-geometric relationships—any deviation in coordinates, colors, or hierarchy is directly reflected in the visual output and is difficult to verify through language descriptions.

The deeper reason lies in the structural difference in "spatial-semantic bidirectional mapping" ability: generating correct Three.js code requires not only understanding JavaScript syntax but also accurately modeling the position (Translation Vector), rotation (Quaternion or Euler Angle), and scale (Scale Factor) relationships between objects in three-dimensional Euclidean space, and understanding the effects of perspective projection and normal direction on lighting calculations; SVG's complex curved paths require precise calculation of the control point coordinates of cubic Bézier curves, where the slightest deviation produces a visually completely wrong shape.

Models trained purely on language pretraining tend to describe space in terms of semantic relationships ("the big one is on top," "the color is darker") rather than understanding semantics in terms of coordinate relationships ("offset 0.3 units on the Y-axis," "opacity: 0.7"), leading to systematic weaknesses in precise graphics output. In contrast, backend agent tasks have a clearer binary success/failure signal (code throws an exception vs. returns a normal result), and this clear reward signal is better suited for reinforcement learning fine-tuning, and more conducive to the model leveraging its self-correction ability through the "execute-error-fix" loop. This structural difference explains why GPT-5.6 scores perfectly on the backend yet consistently lags behind Fable on the front-end—this is not only a matter of training data distribution but also related to the depth of the model's internal representation of "spatial-visual" relationships. Anthropic's Fable's sustained lead on such tasks may be closely tied to its deeper multimodal alignment training in the visual understanding direction, where the fusion approach between its visual encoder and language model gives it a structural advantage in translating visual intent into precise coordinate descriptions.

The creator candidly admits that their benchmark leans toward front-end and visual tasks, so GPT-5.6's total score is somewhat underestimated—front-end development is still not OpenAI's strong suit.

Head-to-Head Comparison with Competitors

On the overall leaderboard, the models scored as follows:

• Fable 5: 88.57% (firmly in the lead)
• Opus 4.8: 87.14%
• Gemini 5.2: 81.43%
• Soul: 78.57%
• GPT-5.5: 38.57%

The most striking comparison comes from the generational gap: Soul's score is nearly double that of the previous-generation GPT-5.5. By any measure, this is a significant capability leap.

Methodological limitations of head-to-head comparison: It's worth noting that KingBench 3, as a single evaluator's self-built benchmark, has a limited sample size (a total of 70 points corresponding to about 10–15 test cases), and the creator states that the test suite leans toward front-end and visual tasks, which systematically lowers GPT-5.6's relative score on backend tasks. Opus 4.8's 87.14%, close to Fable's score, is especially noteworthy—it means Anthropic holds both of the top two spots, consistent with its strategic commitment to multimodal alignment. Gemini 5.2's 81.43% reflects Google's continued progress in code generation and multilingual tasks, closely tied to its technical accumulation in long-context processing (supporting a 1-million-token context window) since Gemini 1.5 Pro. When interpreting any single benchmark ranking, understanding the composition bias of the test set and the evaluator's use case is at least as important as focusing on the ranking numbers themselves. The nearly twofold generational gap between GPT-5.5's 38.57% and Soul's 78.57% is rare within the same product line—a leap of this magnitude usually requires architecture-level innovation (such as an upgrade from Transformer to Mixture-of-Experts) or a fundamental shift in the training paradigm (such as large-scale introduction of synthetic data or multimodal pretraining), rather than simple data scale-up or increased compute. This data point itself warrants deeper investigation and provides important clues for outside speculation about changes in GPT-5.6's underlying architecture.

Objective Assessment: Solid Progress, but Not Revolutionary

Essentially an Incremental Iteration of 5.5

The creator offers a level-headed verdict: GPT-5.6 represents an incremental improvement over GPT-5.5—evolution rather than revolution. The naming of Soul, Tara, and Luna carries a strong marketing flavor, aiming to benchmark against Anthropic's Fable and create the impression of an "entirely new category," but a closer look at actual performance shows it's more like a "stronger, better-priced" iteration of 5.5.

Can It Catch Up to Fable?

The answer is "still a bit short, but already very close." Fable remains the strongest model overall, especially with a clear advantage on front-end and visual tasks. But Soul has already caught up to or even surpassed it in two directions:

• Token efficiency on cybersecurity tasks
• Agentic benchmark performance

On agentic tasks, Soul can already stand shoulder to shoulder with Fable, and Fable's biggest weakness is cost—it is currently the most expensive option. In contrast, Soul's $5/$30 pricing offers a clear cost-effectiveness advantage.

The industry significance of capability convergence: Soul's proximity to Fable on agentic tasks marks that top-tier AI models are entering a "capability convergence period"—that is, the performance gap between the strongest models is narrowing, while pricing and ecosystem are becoming the new competitive axes. The implication for enterprise decision-makers is that, in core enterprise scenarios such as tool invocation, code execution, and multi-step automation, selection criteria are shifting from "which model is smarter" to "which model is cheaper, more stable, and easier to integrate." The pragmatism of OpenAI's pricing strategy this time may hold more strategic significance than the performance numbers themselves. Historically, compute costs have fallen about 50% every 18 months (a Moore's-Law-like curve manifested in AI inference), which means today's "value player" Luna may well become the default choice for mainstream enterprise scenarios a year from now—by then, Soul-level capabilities may already be available at Luna's price. This projection also explains why OpenAI gave the three models independent brand names: when the cost curve keeps declining and the capability curve keeps rising, only a product line with independent branding can let users naturally migrate from low-end to high-end without triggering "self-cannibalization" fears. From a more macro perspective, "capability convergence" does not mean the end of competition, but rather an upgrade in competitive dimensions: when all top-tier models can "do" a certain thing, "how fast it does it," "how much it costs," and "how easily it integrates with existing systems" will become the new variables determining market dynamics—highly similar to the competitive evolution in the PC era from "can it run Office" to "how smoothly does it run Office."

Selection Recommendations

Based on this preview review, here are some reference recommendations (things may still change before the official release):

• Heavy front-end and visual development: Fable or Opus 4.8 remain the top choices
• Backend agent tasks, long-horizon automation tasks: Soul is an excellent choice, at a much lower cost
• Best value-for-money agent model: On a per-unit input cost basis, Luna nearly crushes all competitors

Overall, GPT-5.6 is a strong, reasonably priced upgrade that closes most of the gap with Fable. While it's not as revolutionary as the official marketing suggests, it's genuinely worth close attention—especially for cost-sensitive developers focused primarily on backend agent tasks, the GPT-5.6 series may be one of the most attractive options available right now.

Key Takeaways