AI Credit Deductions Without Breakdown Force Team to Refund $600

A four-person team built an AI Agent product to help sales teams expand their prospect lists. Customers prepay for credits, and the system deducts based on the number of rows processed. On Monday, a customer topped up $2,000. By 4:52 PM, the balance showed only $100 remaining. Tuesday morning, the customer's operations lead sent a sharply worded email, asking whether they had been double-charged and threatening to file a dispute if the matter wasn't explained clearly.

The Trust Crisis Triggered by a Plummeting Balance

The team checked the usage page and found it displayed only one number: 1,900 credits used. No run records, no timestamps, no breakdown of deductions whatsoever. They spent two full days reconstructing the truth from logs: someone on the customer's team had launched a processing task at 1:40 PM on a CSV file with 11,212 rows. The Agent processed every row, and some of them invoked a deep research tool that cost four times as much as standard processing. All operations were legitimate, but when the customer saw the balance plummet from $2,000 to $100, all they saw was a number they couldn't explain.

Why "Legitimate Charges" Look Like Theft

The core issue isn't whether the charges were reasonable — it's that the system couldn't prove they were. From the customer's perspective, a balance drop with no itemized breakdown is indistinguishable from fraud. The team explained the technical details, and the customer believed them, but still demanded a $600 refund as a goodwill gesture. The team had no choice — either refund the money or face a payment dispute.

This exposes a widespread flaw in AI product billing systems: when pricing models become complex (per row, per tool, per depth), a simple balance number simply can't carry enough trust information.

Auditability in usage-based billing systems is a critical property in SaaS and AI product design, referring to whether the system can trace every charge back to the specific event, time, operator, and resource consumption that triggered it. Traditional cloud services (like AWS and Stripe) have long treated auditability as the core of billing infrastructure, because once customers can't verify their bills themselves, disputes can only be resolved through manual customer support or refunds — which is extremely costly. The complexity of AI Agents lies in the fact that a single user action can trigger multiple internal tool calls, each with a different unit price, ultimately merged into a single balance number. Auditability requires the system to model billing events as independent, immutable, context-bearing records from the design stage — not cobbled together from logs after the fact. Without this layer, even if charges are completely legitimate, customers lack the means to verify them, and trust can only be salvaged through refunds.

Similar Cases in the Industry

In their post-mortem research, the team found a blog about billing edge cases, which mentioned that OpenAI had experienced API credits expiring without any warning, with users posting screenshots on social media of balances dropping to zero overnight. This made them realize that even industry leaders can stumble on billing transparency.

API credit expiration and balance zeroing is another common category of billing transparency issues. Many API platforms sell quota in the form of prepaid credits, but credits often come with expiration dates and lack sufficiently prominent reminders before they expire. To customers, credits are real money already paid — sudden invalidation is equivalent to having assets confiscated. The OpenAI case drew attention because it happened to an industry leader, demonstrating that billing transparency isn't a mistake only small teams make — it's infrastructure that the entire industry has broadly neglected during rapid growth. These incidents share the same essence as a plummeting balance: customers can't see the complete lifecycle of their credits — when they were acquired, when they were consumed, when they're about to expire. The solution approach is also the same: turn every change to credits or balance into an event stream that can be queried, subscribed to, and alerted on.

Their Solution

The team then implemented fixes: every run is recorded in the balance changes and shown to the customer; when a customer consumes 50% of their top-up amount within a single day, the system automatically sends an email alert. The core logic behind these measures is simple — turn invisible deductions into a visible, traceable event stream.

Consumption ratio alerting is an effective means of preventing trust crises caused by plummeting balances. Its principle is to intervene proactively before the customer enters a panic zone. Specifically, triggering an email when a customer consumes 50% of their top-up within a day means the system sends a signal when the balance drops from $2,000 to $1,000, rather than waiting until only $100 remains for the customer to discover on their own. The key to an alert email isn't just notifying about the balance — it's including the main sources of consumption during that period, such as which run processed how many rows and which tools were invoked. This way, by the time the customer receives the email, they already have a preliminary explanation, and follow-up inquiries have a clear entry point. Thresholds can be adjusted based on product characteristics — for example, 50% for high-ticket products, 80% for low-ticket products — but the core logic remains the same: turn passive after-the-fact explanations into proactive in-the-moment communication.

What This Means for Your Product

If you're building a usage-based AI product, you need to ask yourself: can customers see the specific operation corresponding to each charge? Can they trace which run, which tool, and which batch of data consumed their credits? If the answer is no, then you're accumulating a kind of hidden debt — every unexplained balance change erodes your customer's trust in you.

The value of this case isn't in its technical complexity, but in the product principle it reveals that's often overlooked: in usage-based billing models, transparency isn't optional — it's the infrastructure of trust.