Claude 4.5 Haiku Real-World Testing Disaster: Coding Capabilities Collapse Across the Board, Crushed by Competitors on Value

Anthropic just released Claude 4.5 Haiku, officially touted as a "cheaper, faster" model that delivers near-Sonnet 4 coding performance at one-third the cost. However, real-world testing from independent evaluators has been nothing short of shocking—this may be one of the worst models released by a major AI lab in recent memory.

The Massive Gap Between Marketing Claims and Reality

In their release announcement, Anthropic stated: "Five months ago, Claude Sonnet 4 was the state-of-the-art model. Today, Claude Haiku 4.5 delivers similar coding performance at one-third the cost and more than twice the speed." They positioned it as a low-cost alternative to Sonnet 4, suitable as a direct drop-in replacement for Claude Code and various applications.

It's worth noting that Anthropic's model naming system borrows literary terms: Opus (large-scale work), Sonnet (fourteen-line poem), and Haiku (the shortest form), implying decreasing model scale. Haiku, as the lightest model tier, corresponds to the smallest parameter count, fastest inference speed, and lowest cost. This tiered strategy is extremely common in the industry—OpenAI has GPT-4o and GPT-4o mini, Google has Gemini Pro and Gemini Flash—the core logic being that many everyday tasks don't require the most powerful model, and using lightweight models can dramatically reduce API call costs.

This sounds great on paper, but real-world testing tells a completely different story.

Basic Capability Tests: Claude 4.5 Haiku Fails Across the Board

Testers ran a series of standardized generation tasks on Claude 4.5 Haiku, and the results were disappointing in nearly every dimension.

Floor plan generation produced layouts that made no sense from any angle—walls were chaotically placed, and for a model claiming Sonnet 4-level performance, this is simply unacceptable.

SVG graphic generation asked the model to create a panda holding a hamburger. While you could vaguely make out a panda shape, the layout and detail work were terrible. SVG (Scalable Vector Graphics) is an XML-based image format that describes shapes, colors, and positions using code. This type of test is widely adopted because it simultaneously evaluates language understanding (comprehending the semantics of "panda holding a hamburger"), spatial cognition (relative positions and proportions of components), and code generation (outputting valid SVG coordinates and path instructions)—making it an effective litmus test for a model's overall capabilities.

3D rendering tasks (3JS Poké Ball) were equally poor. Three.js (abbreviated as 3JS) is a WebGL-based JavaScript 3D graphics library widely used for creating and displaying 3D content in browsers. Asking an AI model to generate 3JS code to render 3D objects like a Poké Ball tests the model's comprehensive understanding of 3D geometry, material systems, lighting models, and scene construction—the model must not only generate syntactically correct JavaScript but also correctly set camera positions, light source parameters, geometry vertex data, and material properties. Any single error in these components will cause rendering failures or visual anomalies.

Chess board generation, web-based Minecraft, and other tests were equally disappointing. CLI tools and Blender scripts wouldn't even run. The only passable result was a "butterflies in a garden" scene, but even that was merely "okay"—nothing impressive.

In the overall rankings, Claude 4.5 Haiku placed 34th and 37th respectively—making it the lowest-scoring new-generation model from any major AI lab. The tester noted that these tests were run more than five times repeatedly, with consistent results.

Agentic Coding Tests: Claude Code Actual Performance Also Falls Flat

If basic tests fail, how about its performance as an agentic coder? Agentic coding is the most cutting-edge application paradigm in AI programming today. Unlike traditional code completion, agentic coding requires the AI model to work like an autonomous software engineer: understanding project requirements, independently planning implementation steps, writing code, running tests, and self-debugging when errors are found—even operating terminal commands and file systems. This mode places extremely high demands on the model—requiring not just strong code generation but also long-context understanding, multi-step reasoning, error diagnosis, and tool-calling capabilities.

Testers used Anthropic's official Claude Code (a tool that lets developers interact with Claude from the command line, allowing the model to directly modify local codebases) for multiple real-world coding tasks:

• Movie Tracker App: Immediately threw a 404 error—the page wouldn't even display
• Go Terminal Calculator: Produced numerous errors with a chaotic layout
• Godot Game Development: Error messages everywhere. Godot is an open-source cross-platform game engine using its own GDScript scripting language (with Python-like syntax), which has rapidly gained popularity due to being lightweight, free, and community-driven. AI-assisted Godot development is an extremely challenging test scenario because GDScript is niche among mainstream programming languages with relatively scarce training data—whether a model can correctly generate Godot project code reflects its coverage of long-tail programming languages.
• OpenCode Repository: Extremely poor performance
• Svelte, Nuxt, Tauri Frameworks: All performed terribly. These three frameworks represent the cutting edge of modern web and desktop application development—Svelte is a compile-time frontend framework that compiles components into efficient native JavaScript during the build phase with minimal runtime overhead; Nuxt is a full-stack framework based on Vue.js providing enterprise-grade features like server-side rendering and static site generation; Tauri is a lightweight alternative to Electron that uses Rust as the backend for cross-platform desktop apps with package sizes one-tenth that of Electron. A model's performance on these mainstream frameworks directly determines its practical value in modern development workflows.

For a model positioned as a "direct Sonnet 4 replacement," this agentic coding performance is nearly unacceptable. Not only does it fail to reach Sonnet 4's level—it may be one of the worst-performing AI coding agents currently available.

Anthropic's Product Line Dilemma: Missing a Lightweight Model

From a product line perspective, Anthropic currently has three model tiers: Opus, Sonnet, and Haiku. Mapped to OpenAI's lineup, Sonnet roughly corresponds to GPT-5 and Haiku to GPT-5 Mini. But Anthropic lacks an ultra-lightweight model equivalent to GPT-5 Nano.

This product line gap means that many enterprises needing lightweight tasks like text summarization or data structuring must either use the relatively expensive Haiku or turn to other providers. Claude 4.5 Haiku appears to have been launched to fill this market gap—targeting enterprise API users rather than consumers.

Value Comparison: GPT-5 Mini and GLM 4.6 Completely Dominate

From a pricing perspective, Claude 4.5 Haiku's value proposition becomes even more problematic. Taking GLM 4.6 as an example, its input/output token pricing is approximately $0.5 to $1.75 per million tokens, while Claude 4.5 Haiku costs roughly three times as much. Yet in actual performance, GLM 4.6 far outperforms Claude 4.5 Haiku.

Recommended alternatives from the tester include:

• GLM 4.6: Exceptional value, performance far exceeding Claude 4.5 Haiku
• GPT-5 Mini: Delivers "crushing" superiority over Claude 4.5 Haiku in comparable tests
• Grok CodeFast: Lower price with stronger coding capabilities

For budget-conscious developers, all three alternatives offer more sensible choices in both performance and cost.

The Deeper Problem: Is Anthropic on a Downhill Trajectory?

The tester raised a pointed but thought-provoking observation: Claude 3.5 Sonnet may have been Anthropic's "lucky strike." That model, through some particular combination of training data, performed exceptionally well in coding. But subsequent iterations—Sonnet 3.7 (essentially the same model with reasoning added), Sonnet 4 (limited improvements), Sonnet 4.5 (regressions in multiple areas)—have all failed to achieve genuine breakthroughs.

The deeper concern is this: as Anthropic scales up and pursues enterprise markets and valuation growth, they may be falling into the trap of "benchmark optimization." This phenomenon is known in the AI industry as "benchmark gaming" or "teaching to the test." The core mechanism is: when model developers become overly focused on specific benchmark scores, they may intentionally or unintentionally expose the model to training data highly similar to test questions, or optimize specifically for test scoring criteria, resulting in artificially inflated benchmark scores while real-world open-ended task performance remains mediocre. Academia calls this "Goodhart's Law"—when a measure becomes a target, it ceases to be a good measure. In recent years, multiple research teams have found that mainstream models' score improvements on classic benchmarks far outpace their actual capability growth in real applications, and this gap is eroding user trust in AI evaluation systems.

Investors focus on benchmark numbers, not actual user experience. Gaming benchmarks to claim frontier progress at low cost is a strategy that pleases investors but ultimately damages user trust.

Conclusion: Is Claude 4.5 Haiku Worth Using?

The release of Claude 4.5 Haiku exposes several critical issues:

1. Severe disconnect between official claims and actual performance—claims near-Sonnet 4 capability but tests far below it
2. Unreasonable pricing strategy—performs worse than GLM 4.6 and GPT-5 Mini while costing several times more
3. Unclear product positioning—neither cheap enough for lightweight tasks nor strong enough for complex coding

For developers and enterprise users, Claude 4.5 Haiku is not recommended as a primary model at this stage. If you need a high-value small model, GLM 4.6 and GPT-5 Mini are both wiser choices. Of course, Anthropic may optimize and fix issues in subsequent updates, but as the current version stands, this is genuinely a disappointing release.

Anthropic once earned the developer community's respect for "not gaming benchmarks and focusing on real-world performance." If they continue sliding down this path, what they'll lose isn't just one model's reputation—it's the credibility of their entire brand.