Claude Opus 4.5 Crushes Humans in Engineering Test: AI Coding Ability Now Surpasses Top Engineers Across the Board

Anthropic's latest release, Claude Opus 4.5, has beaten every human candidate on the company's most rigorous internal engineering test—news that instantly sent shockwaves through the AI community. Meanwhile, Anthropic signed a $30 billion Azure compute deal, OpenAI launched a shopping research mode in ChatGPT, and Google is quietly integrating Notebook LM into Gemini. Let's break down each of these major updates.

Opus 4.5: Crushing All Human Candidates in a Two-Hour Engineering Exam

Anthropic has a famously grueling two-hour timed engineering exam used to screen top-tier engineering candidates. The test requires candidates to complete system design, building, debugging, and tuning under time pressure—evaluating pure technical thinking ability rather than communication or teamwork skills.

According to 2024 Glassdoor reviews, the test is divided into four tiers, requiring candidates to implement a system and progressively add features. Claude Opus 4.5 completed the test under the same two-hour constraint (with multiple runs allowed per question, taking the best result), ultimately scoring higher than every human candidate Anthropic has ever evaluated.

This aligns with what Anthropic CEO Dario Amodei previously stated at Dreamforce—that Claude now writes approximately 90% of the company's code. However, he emphasized that engineers haven't been replaced; instead, they've shifted toward supervising models, correcting the most complex logic, and steering overall project direction.

Comprehensive Benchmark Leadership: It's Not Just About High Scores

Opus 4.5 ranks first in 7 out of 8 programming languages on SWE-Bench Multilingual and achieved a historic high score of 80% on SWE-Bench Verified.

SWE-Bench Background: SWE-Bench is a software engineering benchmark introduced by a Princeton University research team in 2023, specifically designed to evaluate AI models' ability to solve real GitHub issues. Unlike traditional code completion tests, SWE-Bench requires models to read entire codebases, understand problem descriptions, and generate patches that pass unit tests. SWE-Bench Verified is a curated subset with human verification ensuring clear, unambiguous problem descriptions; SWE-Bench Multilingual extends testing beyond Python to multiple programming languages, assessing cross-language generalization. An 80% pass rate means the model can independently solve four out of five real engineering problems—a goal considered far out of reach just a year ago.

But what truly sets it apart is its ability to handle ambiguous bugs—when a bug spans multiple systems, the model doesn't get stuck. Instead, it calmly decomposes the problem, identifies a chain of fixes, and requires almost no repeated prompting.

The "Creative Loophole" in an Airline Customer Service Test

One impressive case comes from TAO 2 Bench—a real-world agent benchmark simulating airline customer service. The scenario involves an anxious customer wanting to modify a basic economy ticket, where the correct answer should be to deny the request since basic economy tickets don't allow changes.

But after reading through the entire airline policy, Opus 4.5 discovered a loophole that even the test creators hadn't anticipated: it first upgraded the customer's fare class (upgrades are permitted), then modified the flight—because once the ticket was no longer basic economy, the change became legitimate. The benchmark flagged this as a "failure" because the creators never expected anyone to think this way.

Anthropic describes this behavior as Claude Opus 4.5's core differentiator: rather than being limited to the most obvious path, it examines rules like a seasoned veteran, searching for creative solutions.

Dual Breakthroughs in Safety and Efficiency

Creative workarounds can pose risks in certain scenarios, which is why Anthropic has invested heavily in AI safety alignment. Opus 4.5 passed the upgraded Petri automated evaluation tool and Gray Swan's highly adversarial prompt injection tests, making it the most difficult model to manipulate with malicious prompts to date.

Prompt Injection Attacks and AI Alignment: Prompt injection is currently one of the most scrutinized attack vectors in AI safety. Attackers embed malicious instructions within inputs, attempting to override the model's original system prompt and force unauthorized operations—such as leaking private data, bypassing content filters, or executing harmful code. Gray Swan is a security company specializing in AI red-teaming, and their adversarial test suite is considered one of the hardest evaluations to pass in the industry. Petri is Anthropic's in-house automated safety evaluation framework that systematically probes model behavior in edge-case scenarios. Passing both tests means Opus 4.5 maintains expected behavioral boundaries even when facing carefully crafted malicious inputs—critical for enterprise-level deployment.

Anthropic calls it "the most robustly aligned frontier model."

Efficiency Parameters: Accomplishing More with Fewer Tokens

Anthropic introduced a brand-new effort parameter in the API, allowing developers to flexibly control the model's reasoning depth:

• Medium effort: Matches Sonnet 4.5's best SWE-Bench score while reducing output tokens by 76%
• Maximum effort: Exceeds Sonnet 4.5 by 4.3 percentage points while still reducing tokens by 48%

Test-Time Compute Scaling: Behind the effort parameter lies one of the most important AI research breakthroughs in recent years. The conventional wisdom was that model capability is primarily determined by training parameter count, but OpenAI's o1 series and Anthropic's extended thinking