GPT-6 Astra's Tax Return Fail: A $2.50 Discrepancy Exposes AGI's Fatal Blind Spot

A Seemingly Flawless AI Tax Filing Demo

On its blog, OpenAI showcased a series of "Computer Use" capabilities for GPT-6 Astra, including one eye-catching demo: having Astra fill out a U.S. federal individual income tax return — Form 1040 — for a relatively simple tax scenario.

On the surface, it was a striking demonstration. The AI could understand the structure of tax forms, read user information, auto-fill various fields, and even complete the tax calculation. This was exactly the kind of "white-collar automation" many people have been anticipating. However, after a careful review, a Reddit user discovered that this professional-looking tax return was riddled with problems — Astra not only made errors but actually "underpaid" the government.

Three Critical Issues in the Details

1. It's Not a Real IRS Form

The first issue the poster flagged: the form used in the demo wasn't the standard f1040 PDF downloaded from the IRS website. It appeared to be an AI-generated HTML-rendered version, with a layout completely different from the official version, hosted on a local web server.

In other words, even if Astra had filled out this "form" perfectly, printing and mailing it to the IRS would most likely result in rejection. This exposes the gap between the demo environment and the real world — the AI was working with a simplified, restructured "sandbox" version, not the actual official document.

2. Unconventional Filing Practices

The second issue is relatively minor but worth noting: nearly everyone fills out tax forms using whole dollar amounts — a practice the IRS permits and recommends. Astra, however, entered precise amounts with decimal points. While not technically wrong, this is notably "unconventional" and clearly deviates from how humans actually do it.

3. The Most Critical Calculation Error

The real problem lies in the tax calculation itself. The poster pointed out: for a taxable income of $36,700, the taxpayer's owed tax should be $4,169, not the $4,165.50 that Astra calculated.

The difference is only $2.50, but the root cause is crucial — Astra used marginal tax rate formulas to calculate the tax, while the IRS explicitly requires the use of tax tables to look up the tax amount for this income bracket.

These two methods produce subtle but real differences. The IRS tax table divides income into small intervals, each corresponding to a fixed tax amount, which doesn't perfectly align with results from a continuous marginal tax rate formula. Astra chose the method that "seems mathematically correct" but violated the actual regulatory requirement.

Why This Small Error Matters

A Serious Question Behind the Humor

The poster wrote with a tongue-in-cheek tone: "Straight to jail; over-claiming $2.50 of a refund is big money." Behind the dark humor lies a serious challenge to the boundaries of AI capability.

Tax compliance is a quintessential "details make or break it" domain. The rules themselves may not be overly complex, but they demand that the executor have a precise understanding of specific provisions and strictly follow "what you're supposed to do" rather than "what seems more logical." Astra's error reveals a core weakness of current large language models: they tend to solve problems using general reasoning while lacking strict adherence to domain-specific rules and the ability to self-verify.

The Absence of Output Verification

The poster explicitly stated that he's a firm supporter of AI and believes AI will take over most white-collar jobs in the near future. But he followed up with a pointed observation:

"If Astra can't even do basic verification of its own output, it's hard to call it an AGI model."

This strikes at the heart of the AGI debate. True intelligence isn't just about producing results that seem reasonable — it's about the ability to self-audit, identify errors, and adhere to established standards. A competent tax professional would know that "you must use the table here, not the formula" and would cross-check the final figures. Astra clearly failed to do this in the demo.

The Gap Between Demo and Reality

This incident also reflects a pervasive issue in AI product demos: carefully curated demos often mask fragility in real-world scenarios. OpenAI chose tax form completion to showcase Astra's ability to handle structured tasks and complex forms. But precisely because tax filing is a domain with extremely clear rules that can be externally verified, any minor error is instantly spotted by professionals.

For AI practitioners and users alike, this case offers several important takeaways:

• Don't blindly trust AI output in specialized domains, especially in scenarios involving law, finance, healthcare, and other high-compliance fields.
• General reasoning ability ≠ domain compliance ability — the former is a model's strength, while the latter still requires dedicated rule constraints and verification mechanisms.
• Self-verification capability is a critical step toward AGI — models need to learn to "check their own work," not just produce an answer.

The Truth Is in the Details

A $2.50 tax discrepancy may seem trivial, but it serves as a mirror reflecting the distance between today's most advanced AI models and true AGI. Astra can understand forms, perform calculations, and generate professional-looking results — impressive in its own right. But its missteps in the details remind us: the real test of intelligence often hides in the rules and verification steps that are easy to overlook.

As AI pushes deeper into white-collar work, ensuring the accuracy, compliance, and verifiability of its output will become a more important question than "can it do the job." After all, when it comes to filing taxes, "close enough" is nowhere near good enough.