Three Costly Lessons from Building AI Agents: Schema Validation, Circuit Breakers, and Retry Strategies in Practice

An indie developer built a Chrome extension where users simply type a sentence like "Create a 10-question onboarding survey for me," and the AI Agent behind it automatically generates a complete form via the Google Forms API. Chrome extensions are browser plugins built on web technologies that can access special Chrome APIs to enhance the browsing experience, while the Google Forms API is part of the Google Workspace developer platform, allowing programmatic creation and management of forms. Embedding an AI Agent in a Chrome extension means the entire reasoning-execution chain runs within the browser environment — the Agent needs to transform natural language instructions into structured requests that conform to API specifications. And in this process, every model call consumes API tokens and incurs costs.

Sounds cool, but on the road to production, the author hit three pitfalls — and each one burned through API budget before they even noticed.

These three lessons may seem trivial, but they reveal the most easily overlooked reality in today's AI Agent engineering: When a model gets stuck in a failure loop, it has no idea what it's doing wrong — it just keeps retrying and burning tokens. Let's break them down one by one.

Lesson 1: Strict Schema Validation Can Actually Drain Your AI Agent's Budget

In AI Agent architecture, the Tool Schema is the core bridge connecting large language models to external APIs. Major model providers like OpenAI and Anthropic all support Function Calling: developers define tool names, parameter types, and structural constraints in JSON Schema format, and the model generates structured output conforming to that Schema to invoke tools during inference. But large language models are fundamentally token sequence generators — their understanding of JSON structure is probabilistic rather than deterministic, and they're prone to type degradation when handling complex or lengthy structures.

In the author's tool Schema definition, the requests field was supposed to be an array. But when processing longer batch tasks, the model would "get lazy" — serializing the entire array into a JSON string instead of producing an actual array structure.

The author's approach was to outright reject this invalid request. From a pure validation standpoint, this was entirely correct. But it was precisely this "correctness" that became the biggest mistake.

The Model Can't See What It Got Wrong

The core issue: The model cannot understand why it was rejected. It only knows the request failed, so it instinctively rewrites the payload — breaking the request into smaller and smaller pieces, trying to "work around" an obstacle it fundamentally doesn't comprehend.

The result was catastrophic: a single "add 20 questions" operation led the model to thrash for 9 rounds, burning approximately 150,000 tokens before giving up and telling the user "the API is broken." LLM API billing is typically based on token count, split into input tokens and output tokens. In Agent loop scenarios, each round requires sending the complete context to the model as input — including system prompts, message history, tool definitions, and interaction records from all previous rounds. As rounds increase, the input token count per round grows linearly or even super-linearly. The 150,000-token consumption happened precisely because each round carried the failure records of all previous rounds, with context snowballing in size. Both user experience and cost control collapsed.

Two Key Fixes

The author offered two low-cost but highly effective fixes:

• If you receive a string, just parse it. Since the string content the model sent is unambiguous, valid JSON, parsing it carries zero risk and virtually zero cost. Rather than dogmatically rejecting it, accept it gracefully.
• When rejecting, point out the actual mistake the model made, rather than throwing the parser's raw error. A message like "Expected double-quoted property name at position 827" is worthless to the model — it can't take any action based on that. But "check whether location is inside createItem, right next to item" states the problem crystal clearly, and the model can correct it immediately.

This detail highlights a universal principle of AI Agent engineering: Error messages are written for the model to read — they must be "actionable," not just "technically correct." This aligns with the core philosophy of Prompt Engineering: instructions to the model should be specific, executable, and unambiguous. Research from Google DeepMind and Anthropic has shown that when error feedback includes clear corrective direction, the model's self-correction success rate can improve several-fold.

Lesson 2: An AI Agent's Round Limit Is Not the Same as a Circuit Breaker

The author originally set a "max rounds" limit to prevent infinite loops, but this limit was set too loosely to catch a stuck model in time.

Why a Fixed Round Limit Isn't Enough

Here's a counterintuitive insight: The fourth identical failure provides no more information than the third. When a model is stuck in repeated failures, giving it more chances won't produce a breakthrough — it will only keep burning money. A generous round limit essentially pays for meaningless retries.

A Circuit Breaker Strategy Based on Consecutive Failures

The Circuit Breaker is a classic fault-tolerance pattern in microservices architecture, first systematically described by Michael Nygard in Release It! and later popularized by Netflix's Hystrix library. Its core idea comes from electrical circuit breakers: when a downstream service fails consecutively up to a threshold, the breaker trips and subsequent requests fail fast without being sent. Traditional circuit breakers typically trigger based on failure rates within a time window, but in the AI Agent context, the author proposes a more precise variant.

The author's new strategy is smarter:

• Stop after 3 consecutive tool call failures. The key word is "consecutive" — any single success resets the counter. This way, a long task that has correctly executed many steps won't be falsely terminated by a cumulative trigger. This design particularly suits Agent work patterns: a complex task might involve dozens of steps where occasionally failing once or twice is normal, but three consecutive failures of the same type strongly suggests the model is trapped in an unrecoverable loop.
• On the final round, explicitly tell the model it's the last round. This way the model explains to the user what happened, rather than "dying silently" and leaving the user with a baffling interruption.

Moving from "fixed rounds" to "consecutive failure counting" is essentially upgrading from "limiting total volume" to "identifying state" — a real circuit breaker cares about whether the system is stuck, not how many laps it has run.

Lesson 3: API Retry Strategy — Retrying Failed Requests Is Almost Always Wrong

The third lesson concerns retry strategies for network requests, and the author's conclusion is clear-cut and worth remembering for all AI Agent developers.

Never Retry HTTP Error Responses

The HTTP protocol defines a clear status code system: 4xx indicates client errors (e.g., 400 Bad Request, 429 Rate Limited), and 5xx indicates server errors (e.g., 500 Internal Server Error, 503 Service Unavailable). In traditional web development, retrying 5xx errors with Exponential Backoff is standard practice because server errors are usually transient. But the billing logic of AI model APIs completely changes this equation.

When you receive an HTTP error response (say, 4xx or 5xx), it means the model has already run and you've already been charged. Even with a 500 error, model inference may have completed and incurred billing. Blindly retrying at this point only gets you double-charged. The request already reached the server and was processed — there's no safety guarantee in retrying.

The Only Exception Worth Retrying: Connection Not Established

But there's one critical exception: when fetch throws a TypeError rejection, it means no response headers arrived — in other words, the request never reached the server and nothing was billed. In JavaScript's fetch API, when the network connection itself fails (DNS resolution failure, TCP connection timeout, TLS handshake failure, etc.), fetch throws a TypeError rather than returning a Response object. This distinction is crucial: a TypeError means the HTTP request never left the client or never reached the server's application layer, so no server-side processing or billing could have occurred.

This is exactly what a "connection failure" looks like on the user's end. For users, this is the most common scenario that most needs graceful handling. Therefore, this is the one case where retrying is safe — and the one case most worth retrying. This is also the practical application of Idempotency in the AI API context: retrying is only safe when you can confirm the request was never processed.

The criterion is crystal clear: Did the request actually reach the server? If it did, don't retry. If it didn't, retry. This simple dividing line avoids double billing while still providing fault tolerance for genuine network jitter.

Core Takeaways for AI Agent Developers

Looking at all three lessons together, they point to the same deeper theme: Error handling logic for AI Agents is fundamentally different from traditional software.

In traditional programs, strict validation, fixed retry limits, and error-on-failure are all impeccable best practices. Error handling is built on the assumption that "the caller can understand the error": developers see a stack trace and locate the problem; programs receive error codes and execute corresponding recovery logic. But AI Agents introduce a fundamental variable: the decision-maker in the call chain is a probabilistic model that understands error messages through natural language and generates repair strategies based on statistical patterns. In Agent scenarios:

1. The model is a black box that "can't see the context." Your error handling strategy must assume the model cannot understand technical details, so error messages need to be "translated" into language the model can act on. This is precisely why more and more Agent frameworks (like LangChain, CrewAI) are beginning to build in structured error feedback mechanisms — translating raw exception messages into model-friendly natural language descriptions.
2. Every loop iteration costs money. Runaway retries aren't a performance issue — they're direct financial losses. Circuit breaking must be based on state assessment of "whether it's stuck," not crude count limits.
3. Tolerance is cheaper than strictness. If the input can be parsed unambiguously, accept it. Don't pay the expensive price of retries for the sake of validation purity.

For any team building production-grade AI Agents, these three lessons — born from real production environments and "discovered only after real money was burned" — deserve a place in your engineering checklist. An Agent's reliability often depends not on how smart the model is, but on how you design its behavior when it fails.