OpenAI 6.2 Astra Hands-On Review Gone Wrong: Impressive Benchmarks, But Developers Say It's Unusable

6.2 Astra leads benchmarks and costs less, but in real workflows it behaves erratically, explains poorly, and produces convergent ideas.
OpenAI's 6.2 Astra significantly outperforms Claude Fable 5.2 on benchmarks like Agent's Last Exam at about 5% of the cost, with across-the-board price cuts — textbook launch numbers on paper. Yet early testers report the opposite: the model lacks a sense of knowledge boundaries in long tasks, throwing incomprehensible internal implementation questions at users and self-blocking; resource management is out of control, consuming memory, breaking permissions, and filling disks; explanations of complex tasks miss the point, forcing testers to switch to other models for summaries; creative tasks show clear "regression to the mean," with multiple testers independently getting outputs with the same name and theme. Individual highlights in games and 3D but integration struggles, with best practices trending toward model combinations. The article concludes: benchmarks measure capability ceilings, but behavior determines retention value in real workflows.
OpenAI's 6.2 Astra has delivered a rather impressive report card on the benchmark leaderboards, yet a group of developers who participated in early testing have given diametrically opposite assessments. After several weeks of in-depth experience, a YouTube tech creator even described his feelings with the phrase "I hate using this model."
That contrast itself is worth a closer look. Why would a model that leads across the board in benchmark tests and costs less make people want to tear their hair out in real workflows?

The Benchmarks Really Deliver, and the Price Really Is Lower
Let's start with the data. According to OpenAI's official figures, 6.2 Astra scored 78.3 on Agent's Last Exam, a multi-step professional task benchmark, 5.5 percentage points higher than Claude Fable 5.2, at only about 5% of the latter's cost. On Terminal Bench Science, it likewise beat the previous generation 6 Astra by 9 percentage points, at less than one-third the cost.
The creator's own testing confirmed this: Astra cost roughly half of Fable to run the full test suite, while the lighter 6.2 Sol was even cheaper at nearly one-eighth. On pricing, Astra dropped from $10 to $8 per million input tokens and from $50 to $40 for output, with an even more significant reduction in cached input costs.
Looking at these numbers alone, this should be a textbook-grade launch.
Agent's Last Exam is a multi-step professional task benchmark for agents. Unlike static knowledge Q&A, it requires the model to continuously execute multiple interdependent operation steps in a simulated environment (such as calling tools, retrieving information, modifying files) to ultimately accomplish a composite goal. Scoring typically considers task completion, step correctness, and resource consumption. This type of benchmark is closer to real development workflows, but still struggles to cover behavioral stability issues in long-horizon tasks.
Terminal Bench Science focuses on the model's scientific computing and engineering operation capabilities in a terminal environment, testing whether the model can correctly use command-line tools for data analysis, environment configuration, and other tasks. It measures "whether it can do it right," not "whether it's hassle-free to use."
The Problem Is "How to Use It," Not "Whether It Can Do It"
The real point of divergence is the interactive experience. The creator repeatedly emphasized that Astra is genuinely top-tier in individual capabilities like math problems, 3D environment generation, and computer operation — "the computer use capability is absolutely explosive, absurdly fast." But once it enters long tasks, problems emerge densely.
The most typical is the lack of a sense of knowledge boundaries. The model performs reasonably well when managing multiple sub-agents, but will suddenly throw out an internal implementation detail question that an ordinary user simply cannot understand, then sit there waiting for an answer. The creator described an experience having the model create a new level for Outer Wilds: two hours later, the model stopped and asked him to compare the write permissions of certain file paths — paths he had never seen before.
"I've never encountered a model like this. It can't distinguish what information it knows itself, what sub-agents should know, and what the human prompter should know."
Resource management is equally out of control. During operation, the model heavily consumes memory, breaks permission settings, fills the disk with randomly generated useless files and refuses to clean them up. The creator had to invoke another model to free up space before he could continue working.
The sub-agent mentioned in the article is a multi-agent architecture: the main model decomposes a complex task, spawns multiple sub-agents to execute subtasks separately, and then the main model aggregates the results. This pattern can improve parallel efficiency, but it also places higher demands on the model's "sense of knowledge boundaries" — the main model must be clear about what information belongs to itself, what belongs to sub-agents, and what needs to be confirmed with humans. Once the boundaries blur, the model may seek input from the wrong entity, causing the task to stall.
Output Readability: Explanations Are Harder to Parse Than Code
A frequently mentioned pain point is "explanation ability." The model can write code of acceptable quality, but when explaining what it did, it dives headlong into details and actually drifts away from the facts.
The creator gave a specific example: work tree creation was slow on a certain machine, and Astra wrote four long paragraphs analyzing why the package manager was slow, meandering around the filesystem, yet never stated the core answer — that VPS's storage path was much slower than other machines. When the same question was given to Opus 5.5, it pinpointed the cause in one sentence.
This situation forced the creator to develop a habit: after a long task completes, switch to another model and have it summarize what Astra actually did. "Without this step, aside from simple bug fixes, it's basically impossible to parse its output."
Work tree is a concept in Git version control, referring to multiple independent working directories derived from the same code repository, allowing developers to work in parallel on different branches without interference. Creating a work tree requires copying or linking repository objects, and its speed is significantly affected by underlying storage I/O performance. In the article, the VPS's slower storage path is precisely the root cause of slow work tree creation, yet the model bypassed this core fact and gave an analysis that missed the point.
Regression to the Mean: Why Creativity Always Collides
A more subtle problem appears in creative tasks. The creator asked Astra to design a new planet for Outer Wilds and got a water world with themes of tides, music, and bells, named Bellwether. He initially thought this was quite creative, until he discovered that his podcast partner, using an almost identical prompt to have the model add a level to Majora's Mask, also got a water-themed, music-themed level with a bell core, also named Bellwether.
Subsequently, other testers reported similar phenomena: someone had Astra make a spy puzzle game set at Bellwether Station; someone had it create a monster, and the monster was also called Bellwether.

"I hate this idea — what I initially mistook for creativity is actually a set of hardcoded names and a collection of vague concepts."
Regression to the mean is originally a statistical concept, referring to the phenomenon where extreme results tend toward the average in subsequent observations. In generative models, it is borrowed to describe a behavioral pattern: when prompts lack sufficient constraints, model outputs converge toward high-frequency concepts, naming conventions, and themes in the training data. Multiple testers independently getting the name "Bellwether" and water/music themes indicates that the model is not "creating" but sampling from high-probability regions. This suggests to developers: if genuine diversity is needed, constraints must be actively imposed through prompts or sampling parameters.
Games and 3D: Individual Highlights, Integration Struggles
In game development scenarios, Astra's performance is mixed. The Pokémon ROM hack it generated runs, but sprite images have cropping issues and inconsistent pixel styles; the model cannot generate precise pixel art at specified resolutions. In an attempt to HD-remaster Final Fantasy VII, AI-generated textures flickered frequently and felt disconnected from the original art style; the creator felt some scenes actually looked better in the original.

However, in purely visual displays like aquarium games, the model's output was "one of the best-looking things I've seen in a browser." OpenAI also acknowledges that the new model has made notable progress in front-end construction for websites, games, and 3D design.
A representative conclusion is that the best results often come from model combinations — using OpenAI models for asset and 3D generation, and Anthropic models for polishing playability and details. When the creator had Astra do level design and Fable take over to optimize the version, the water effects, model fitting, and overall experience were all significantly better.

A "Complaint Letter" to OpenAI
The most candid part of the video is the creator publicly sharing an internal feedback letter he wrote to OpenAI during the early testing phase. In the letter, he affirmed the model's strengths: excellent code review ability, almost never missing issues and rarely fabricating; occasionally proposing genuinely novel solutions, such as a reverse proxy approach estimated to save $25,000 per month; 3D and game content generation at world-class level.
But the rest of the letter was worded quite directly. He described the model's performance when migrating a project from TypeScript to Rust: it ran for several days without any substantive improvement, and actually made the compiler slower; after switching back to Opus, the latter landed a large number of changes within two days with noticeable performance gains.
He also predicted the post-launch public opinion trajectory: day one, the whole internet marvels; day two, people start noticing high consumption and low output; day five, discussions emerge about "is Sol secretly better"; two weeks later, the model is forgotten. He admitted he might be overreacting, but insisted every word was a real experience.
To OpenAI's credit, before release it adjusted the system prompt based on this feedback, adding rules like "rewrite the final answer for the reader" and relaxing mandatory font requirements. The creator considered these changes helpful but insufficient.
What This Launch Reveals
The 6.2 Astra case reveals an emerging industry reality: benchmarks measure the upper limit of a model's capabilities, while what developers face daily is the average of the model's behavior. A model that can set records on hard problems will have its value in real workflows greatly diminished if it frequently stalls in long tasks, produces hard-to-parse output, and manages resources crudely.
The creator concluded with a rather pragmatic expectation: he hopes OpenAI's next version adopts a "Snow Leopard"-style release approach — not pursuing higher benchmark scores, but fixing behavioral quirks to make the model more stable and predictable. "Even if the benchmark scores don't go up, as long as it's more usable, I'm willing."
For developers currently making selections, this may be the most noteworthy takeaway: benchmarks determine whether you'll try it; behavior determines whether you'll keep it.
Related articles

Why Git Is Essential in the AI Era: From Version Control to Agent Collaboration
What's the difference between Git and GitHub? How do you manage changes with Issues, branches, and Pull Requests after AI Agents write code? Based on NetworkChuck's Git tutorial, this article walks through the complete flow from git init to conflict resolution and analyzes why version control has become an essential skill in the AI era.

GTA VI Coffee System Details Revealed: Hot Coffee Can Be Splashed on People?
Vergecast podcast reveals GTA VI's coffee system: players can brew in apartments, buy from vending machines, order in restaurants, and even splash hot coffee on others. This detailed recreation of daily interactions continues Red Dead Redemption 2's immersive design and sparks discussion on game detail versus gameplay balance.

Breaching Anthropic's Slack in 12 Hours: The AI Moat Is Underestimated
AI security researcher Paul Christiano reveals that unsophisticated attackers breached OpenAI's code repo in days and Anthropic's Slack in 12 hours—the actual lead window may be just 12 hours.