GPT-6 Sol vs. Legacy Spaghetti Code: Can It Dethrone Astra?

A Stress Test Against Legacy Spaghetti Code

The newly released GPT-6 Sol has sparked intense discussion in the AI community, with one question at the center of it all: can it dethrone Astra from its top-tier position? Rather than following a standard benchmark route, this test took a different approach — throwing Sol directly into the chaos of a real-world legacy codebase to see how it handles repairs under messy, real-world conditions.

On the AA comprehensive capability leaderboard, Sol shows a modest improvement over its predecessor Terra in the Max tier, but still trails Astra by a noticeable margin. This baseline context matters — it sets the right expectations for interpreting the test results: Sol isn't here to dominate; it's here to prove its value-for-money and practical utility.

34 Functional Tasks, One Shot Each

The test used a community-contributed "spaghetti code" benchmark consisting of 34 functional tasks, covering a range of classic "hidden bugs": a tool that clearly fails but displays a success message on the UI; reply references pointing to the wrong object when switching between drawings; settings changed mid-run that don't affect subsequent behavior. These are exactly the kinds of problems that make real-world development so painful.

Tasks were divided into four difficulty tiers — Easy, Medium, Hard, and Finale — with one critical rule: you cannot break functionality that was previously working, or face a 5-point deduction. This rule cuts right to the core challenge of AI-based code repair — many models can fix the target issue while inadvertently introducing new breakage.

The test conditions were strict: one prompt, one repair round, no follow-up hints allowed, and the model had to get it right in a single pass. This puts Sol's "one-shot" capability under a magnifying glass. Execution used the Codex model, with Sol's reasoning level dialed up to maximum.

What is "Spaghetti Code"? "Legacy Spaghetti Code" is developer slang for codebases that have been iteratively patched over time without proper standards — tightly coupled, poorly documented, and difficult to reason about. Typical characteristics include logic scattered across multiple files, chaotic variable naming, and hard-to-trace side effects, where fixing one thing often breaks another. Using spaghetti code as an AI benchmark is far more representative of real engineering environments than LeetCode-style algorithm problems or clean unit tests. It demands not just local logic comprehension, but global context awareness — the ability to make surgical fixes without disturbing existing behavior. That's exactly why the "don't break what works" penalty rule is so critical: it directly measures a model's precision in navigating complex dependency chains, not just its raw code generation ability.

Clear Debugging Logic, Final Score: 40.48%

Sol's debugging process demonstrated solid engineering instincts. It started from the working directory and laid out a structured investigation covering chat handling, history storage, and runtime configuration — a reasonably complete logical chain.

The final result: 14 out of 34 tasks passed, with a raw score of 40.48% before deductions. One highlight worth calling out separately — Sol successfully passed two out of the three Finale-tier tasks, the hardest in the set. This suggests Sol has genuine capability when dealing with high-complexity problems, rather than only excelling at simpler tasks.

The Gap with Astra: Stability Is the Weak Spot

Plotting Sol on the spaghetti code benchmark leaderboard alongside other models, many of which score in the teens to mid-twenties, Sol currently ranks fourth based on its recorded entry. It's worth noting that the leaderboard entry and this specific test run are not the same session — the comparison is for reference only.

Even so, claiming this single run is enough to challenge Astra's top position would be a stretch. More critically, this evaluation surfaced a problem that can't be overlooked — regression. While Sol fixed some issues, it also disrupted previously working functionality, triggering the penalty rule. For production-facing development, this kind of "fix one, break another" instability is a hard flaw to overlook.

Pricing Is Its Biggest Weapon

Where Sol truly stands out is in its pricing. Based on standard API rates for short contexts, both input and output token prices are just 20% of Astra's.

Compared to its predecessor Terra, input pricing is on par, while output is actually slightly cheaper. Of course, these are per-token rates — actual costs depend on usage volume. But at this price point, achieving these results in a single repair round is genuinely compelling for everyday development use.

In other words, Sol's value proposition isn't "the best" — it's "good enough, at a fraction of the cost." When you can get substantial repair capability for one-fifth the price, cost-efficiency itself becomes a competitive advantage.

Why Token Pricing Matters More Than It Looks AI model APIs are typically billed per token — input tokens cover the text sent to the model (code, prompts, context), and output tokens cover the model's generated response. In code repair scenarios, token consumption can be substantial — a full codebase context plus detailed repair instructions can easily run tens of thousands of tokens per call. So even if the per-token price difference seems like number games in isolation, the cumulative cost difference in real projects can be dramatic. Sol's input/output prices at 20% of Astra's means a team can run five times as many repair iterations on the same budget, or cut AI call costs to one-fifth for the same workload. This is especially significant for automated development pipelines that make frequent API calls.

Verdict and What Comes Next

Here's the current takeaway: Sol can handle spaghetti code, but its post-repair stability still needs work. Its debugging logic is clear, its Finale-tier performance is impressive, and its pricing is a genuine advantage — all strong points. But the regression problem remains its biggest barrier to challenging the top tier.

One question left for the next round: if the failed tasks and regression issues are fed back as follow-up prompts with two more chances to fix them, can Sol come back and clean everything up? That would determine its true ceiling in real development workflows. For now, at minimum, it has proven itself a high-value, cost-efficient option worth keeping a close eye on.