Opus 5 Real-World Testing: Cheaper Yet Stronger — Why It's Now My Default Coding Model

Anthropic quietly released Opus 5 on a Friday afternoon, but the buzz it generated was anything but quiet. A developer and YouTuber who extensively uses frontier models from various companies reached a conclusion after a full day of hands-on coding: Opus 5 might be the only model you need right now. This sounds counterintuitive — because the same company has a larger, more expensive model (codenamed Fable, a higher-tier version of Opus) that actually trails behind in some benchmarks. This article breaks down the technical logic behind this "cheaper yet better" paradox.

Counterintuitive Benchmarks: Cheaper Yet Stronger

In nearly every available benchmark, Opus 5 ranks at or near the top, even beating its sibling Fable 5 and competitor 5.6 Sol. The key is pricing: Opus 5's token pricing is $5 per million input tokens and $25 per million output tokens, exactly half of Fable 5 ($10/$50).

A "token" is the basic unit of text processing for large language models, roughly equivalent to 3/4 of an English word or half to one Chinese character. All major AI APIs charge based on total input and output tokens, with output typically costing several times more than input because generation is computationally more expensive.

Looking at specific results, Opus 5 shines across multiple dimensions:

• Frontier Code (a benchmark measuring code maintainability and mergeability): jumped from a previous high of 34.4% to 43.3%. Frontier Code fundamentally differs from traditional benchmarks like HumanEval and MBPP that only test functional correctness — it evaluates whether code follows a project's architectural conventions, naming standards, and modular principles, and whether it could pass a senior engineer's code review and be merged into the main branch. The leap from 34.4% to 43.3% means AI-generated code went from roughly one-third directly mergeable to nearly half, which has far greater practical significance for engineering efficiency than a few percentage points on functional benchmarks.
• ArcAGI3 (a pattern recognition test that penalizes excessive reasoning steps): scored approximately 30 points, while no model could exceed 1% when the benchmark first launched. Designed by François Chollet, ArcAGI measures AI systems' abstract reasoning and pattern generalization abilities. Test problems present grid-based input-output transformation pairs, requiring models to infer transformation rules and apply them to new inputs. Version 3 added a penalty mechanism for reasoning efficiency — not only must answers be correct, but models can't brute-force their way through massive reasoning steps to find answers. This makes it an important indicator of whether a model truly "understands" problems rather than "brute-force searching."
• BrowseComp (agentic search): 90.8, nearly tied with 5.6 Sol's 90.4
• Humanity's Last Exam (with tools): excellent performance

But here's an interesting recurring pattern: When Fable is tested purely on knowledge, it's stronger; when tested on the ability to use available tools to accomplish actual work, Opus 5 often overtakes or matches it. In pure knowledge scenarios like tool-free HLE and HealthBench, Opus 5 still trails the larger model.

The Price Advantage Isn't That Large

Don't be misled by "half price." While the per-token cost is 50% cheaper, Opus 5's token efficiency is notably lower — according to Artificial Analysis data, Fable 5 uses about 33k tokens per task while Opus 5 needs 37k. Token efficiency refers to how many tokens a model consumes to complete the same task; lower efficiency means the model tends to generate more verbose responses or requires more intermediate reasoning steps. This not only increases costs but also depletes the context window faster (the maximum total token count a model can process in a single conversation), affecting coherence in long conversations or complex tasks. All things considered, the real-world cost savings are closer to 20% to 25%, not the apparent 50% (per-task cost: Fable ~$2.75, Opus 5 ~$2.03).

Three Cross-Reviews Reveal a "Middle Ground" Personality

The most convincing evidence isn't benchmark scores, but a carefully designed controlled experiment. The author used identical prompts to have both Opus 5 and Fable 5 develop implementation plans for the same technical task, then had them cross-review each other's plans.

The result was surprising: Both models thought the other's plan was better. Fable gave Opus's plan near-perfect scores (multiple 10s), stating Opus was significantly superior in key areas; while Opus scored Fable lower across the board (averaging ~8), yet still said Fable's plan was better overall.

Even more surprising was the third-party judge: the author had 5.6 Sol blind-evaluate both plans (labeled O5 and F5, with Sol unaware of which was which). Sol gave Opus's plan 8.3 and Fable's only 6.0. In a situation where everyone (except Opus itself) thought Opus's plan was better, Opus humbly suggested publishing Fable's plan as the foundation.

This validates an elegant metaphor: if Fable is a "deeply thoughtful, eloquent wise owl" and 5.6 Sol is a "Rottweiler that locks onto problems and won't let go," then Opus 5 is the perfect middle ground between the two — combining Sol's execution power and instruction-following with the distinctive "taste" of Anthropic's premium models.

Distilled from Mythos: Balancing Safety and Capability

Why can a smaller model outperform a larger one on benchmarks? The author's explanation is distillation.

Distillation is a classic model compression technique in deep learning, systematically proposed by Geoffrey Hinton and colleagues in 2015. Its core idea is to have a smaller "student model" learn from the output distribution of a larger "teacher model," rather than directly learning from hard labels in the raw training data. The teacher model's softmax outputs contain rich inter-class relationship information (called "dark knowledge") — for example, it knows not just what the correct answer is, but which wrong answers "almost got it right." By mimicking this probability distribution, the student model can achieve comparable or even superior performance on specific tasks with far fewer parameters than the teacher.

Think of the largest model, Mythos, as a bowl filled with all foods — containing plenty of high-quality content but also some dangerous capabilities (like exploit development and other "double-edged" skills). When training Opus 5, Anthropic essentially filtered this bowl for the parts they actually wanted: less food overall, but everything remaining is the cream of the crop.

This is the "teacher-student" distillation relationship: the best teacher isn't someone who never made mistakes, but someone who made every mistake and can guide students around pitfalls. Mythos, the model with the most information, was used to precisely inject correct knowledge into the smaller Opus 5. In the LLM era, distillation has become a critical technique for commercial deployment — OpenAI's GPT-4o Mini, Google's Gemma series, and others all heavily use this technique to balance performance and cost.

On the safety front, distillation brought a substantive breakthrough. According to Anthropic, Opus 5 is their best-aligned model to date, with the lowest deception rate and greatest resistance to misuse. The key difference is in mechanism:

• Mythos and Fable are essentially the same model, differing only in a "classifier guard" placed in front that controls what requests can enter and what responses can exit
• Opus 5 has safety constraints baked directly into the model itself

These two safety paradigms are fundamentally different. The bolt-on "classifier guard" approach places independent safety classifiers at both the input and output ends of the model, performing real-time review and filtering of user requests and model responses. This method is simple to implement and flexibly adjustable, but the model itself hasn't actually "learned" to refuse dangerous behavior — the safety layer and capability layer are decoupled, making it vulnerable to carefully crafted prompts that bypass it (so-called "jailbreak" attacks). Training-time alignment uses techniques like RLHF (Reinforcement Learning from Human Feedback) and Constitutional AI to embed safety constraints into the model weights themselves, making them much harder to bypass. However, this has historically caused models to become overly conservative — a performance loss the industry calls the "alignment tax."

Historically, such built-in restrictions severely damaged a model's practical coding ability (especially debugging), but Opus 5 maintained safety while paying almost no capability cost — it's still good at finding vulnerabilities (helpful for fixing bugs), but its ability to exploit vulnerabilities has been significantly weakened. The author calls this "a lobotomy in the right direction."

Real-World Experience: Strengths and Annoying Edge Cases

The author coded with Opus 5 for an entire day straight, completing numerous PRs that would actually be merged into codebases.

Subscription quota consumption is the most tangible advantage. The author maintains three Fable subscription accounts simultaneously because Fable has been removed from the $20 Pro plan and only gets half the weekly quota on higher-tier plans. The comparison:

• Using Fable: a day's work burns through ~1.5 weekly quotas
• Using Opus 5: the same intensity burns only ~12% of a weekly quota

That's a four-to-five-fold difference. This is partly because Opus itself is cheaper, and partly because Anthropic allocates more generous quotas for the lower-demand Opus.

Additionally, Opus 5 supports Zero Data Retention (ZDR), while every Fable/Mythos request is audited and logged by Anthropic. Zero Data Retention means the AI service provider commits to not storing or logging users' API requests and response content. This is critical for enterprise users because under regulatory frameworks like HIPAA (healthcare), SOC 2 (finance), and GDPR (EU), sending customer data or proprietary code to an AI system that records and audits it could directly violate compliance requirements. Many large enterprises were previously forced to choose self-hosted models or forgo cutting-edge closed-source AI for this reason — Opus 5's ZDR policy now opens the door for them.

Lingering Traces of Being "Dumber"

As a smaller model, Opus 5 occasionally reveals its limitations. The author encountered an annoying case: while handling a task, Opus 5 opened a browser on its own three times (not even the default browser), interrupting the author's work.

The subsequent conversation was even more interesting — when the author confronted it, the model first blamed the CLI tool, then honestly admitted fault after being called out: "Three attempts to open a browser you never asked for… that was my own extra task, not worth interrupting you."

On factual knowledge, Opus 5 is also slightly inferior to Fable 5 due to its smaller size. It scored 31 on the AI Omniscience benchmark (measuring a model's ability to avoid lying when it doesn't know the answer), an improvement over Opus 4.8 but still showing a tendency to "lie to make answers simpler." This phenomenon is called "hallucination" in the AI field, where a model generates factually incorrect content with high confidence. The root cause is that language models are fundamentally probabilistic prediction engines — they predict the "most likely next token" rather than the "most correct next token." Larger models, having encountered more data during training, can usually better distinguish the boundary between what they "know" and "don't know." If hallucination is your core pain point, Fable 5 remains the least likely to fabricate information.

Selection Guide: How to Choose Among Three Models

Based on real-world testing, the author provides clear selection recommendations:

5.6 Sol: Pure Tool Utility

Best for scenarios where you just want to "give commands, get results" — side projects, one-off scripts, automation tasks, AI assistants (extremely fast responses). It will solve problems at all costs, but code quality is questionable and it tends to write too much code. Its biggest advantage is token efficiency: the author found one Codex account sufficient, and even after accidentally spawning 32 agents and using it intensively for days, only used 30% of a weekly quota.

Fable 5: Code with Taste

Fable writes more aesthetically pleasing code that better conforms to system design, has stronger frontend capabilities, and "knows more" — more likely to find good answers for obscure bugs or niche platforms. It excels at task orchestration. Downsides are high cost, slow speed, and lack of thoroughness: it often rushes to "cleverly work around problems" rather than solving them properly.

Opus 5: The Default When You Don't Want to Worry

The reason to use Opus 5 is simple: you don't want to agonize over the above trade-offs anymore. It follows instructions, stays focused on the task, is extremely careful (even slightly self-doubting, which actually steers it in the right direction), and feels more like an OpenAI model than any previous Anthropic model. The author calls it "like the child of these two models."

Notably, Anthropic rushed out a document titled "New Context Engineering Rules for Claude 5 Models," with the core message being "you no longer need to repeatedly and verbosely constrain the model" — past Anthropic models loved guessing what you "really wanted," requiring constant reminders of "don't do that." Opus 5 is the first Anthropic model that truly "does what you say, asks when uncertain." This shift is significant in prompt engineering: it reduces the cognitive burden of user-model collaboration, allows system prompts to be more concise, and reduces "prompt pollution" problems where over-constraining actually causes abnormal model behavior.

Don't Just Trust YouTubers — Test with Your Real Tasks

The author's final verdict: Fable is still their favorite model, but Opus 5 delivers a "genuine Fable flavor" at lower cost and with a slower but far more quota-efficient approach. It often catches details Fable misses, writes code more likely to actually work, and eliminates the need to constantly switch between Sol and Fable as a compromise.

The most valuable advice: Don't blindly trust evaluations from YouTubers and journalists. The author even publicly called out a tech publication for getting the common sense of "smarter models use more tokens" backwards. The truly reliable approach is to feed your own real tasks to multiple models simultaneously, then have them cross-review each other or introduce independent threads for auditing — comparing different models' outputs on the same task is the best way to understand AI model differences.

Key Takeaways