5 Core Principles for Building Reliable AI Agents

When AI agents move from demos to production, reliability shifts from being a technical metric to the make-or-break lifeline for success. While building Kept, an open-source e-commerce after-sales agent, developer Dragos distilled a set of design principles centered on "reliability first" that keep the Agent Loop running stably in real-world scenarios. These lessons offer direct reference value for teams pushing agents into production.

What Is an Agent Loop

An agent is essentially a loop: send a message to the LLM, execute the tool calls the model requests, return the tool results to the model, and repeat until the model decides the task is done and is ready to give a final answer. Each model invocation is called a round, and each round, tool call, and tool result is persisted to the conversation history, then sent in full in the next round. A turn corresponds to one customer message input and one result output, composed of one or more rounds.

The author recommends starting with Amp Code's "How to Build an Agent" tutorial to understand the basic structure. But what truly sets Kept apart from an ordinary demo is a series of design decisions the author made after putting on "reliability glasses."

The technical essence of the Agent Loop is engineering a stateless LLM call into a loopable, persistable stateful flow. The LLM itself has no memory: every call only sees the full message array passed in that time, so the agent's "memory" comes entirely from externally persisted conversation history. A round is one closed cycle of "request the model → execute tools → return results"; a turn is the outermost customer interaction, which may contain multiple rounds until the model stops requesting tools and gives a final answer. Understanding the hierarchical relationship between round and turn is crucial, because reliability constraints like rate limits, budgets, locks, and atomic writes often act at different levels—for example, "only one request per conversation at a time" constrains the turn, while a token budget may constrain a single round. Clarifying this hierarchy gives a unified coordinate system for the subsequent design of the state machine, host, and error handling.

Principle One: Draw the State Machine Before Writing Code

Reliability lens: Do I know all the possible ways a turn can end? Or am I waiting for production to tell me?

The first key decision is to define the state machine before writing any code. This provides a structured way to think about edge cases—which events can fire in which state of the loop, and which terminal states cover those edge cases.

The author's method has four steps: list all events (who is waiting, what each situation can return), define states (where the loop waits), build a state machine table (events as rows, states as columns), and define terminal states (flag every cell where a model call can no longer be initiated). The result is a complete transition table where every impossible transition is annotated with its invariant and enforcer.

This state machine translates directly into code: terminal states become union types, the table becomes a switch statement on the model response, and impossible cells become preconditions in function-header comments.

Principle Two: Separate the Loop Runner from the Host

Reliability lens: What does my agent loop take for granted? Who guarantees those assumptions?

The loop has a set of invariants from the state machine, the most important being "only one request per conversation at a time." This requires a conversation lock, but folding that logic into the loop runner would make it overly complex and hard to test.

The author's approach is to turn these invariants into preconditions of the loop runner and delegate responsibility for guaranteeing them to a new component—the Host. The host is responsible for: reading environment configuration, initializing dependencies, resolving conversation identity before acquiring the lock, persisting the turn to conversation history, ensuring the trace ends in a finally block and is exported, and wrapping the response envelope the HTTP handler needs.

After separation, the loop runner keeps clear responsibilities: receive a customer message and history, drive the model and tools until a terminal state is reached, and return the result along with the history to be persisted. This architecture is equally friendly to evaluation (Eval)—the eval runner is just a second host, composing the same service with its own set of dependencies, with each case running the unmodified loop in a world built specifically for it.

Preconditions and invariants are the core concepts of this architecture. An invariant is an assertion that must hold at all times, such as "only one in-flight request per conversation at a time"; a precondition is an assumption that must already be satisfied before a function is called, such as "the caller has acquired the lock for this conversation and resolved the conversation identity." Shoving invariant-maintenance logic into the loop runner makes it responsible for both business orchestration and concurrency and lifecycle management, forcing tests to construct real dependencies like locks, storage, and traces. Once the responsibility for guaranteeing invariants is moved out to the host, the loop runner's input contract becomes pure: one message plus a history, and its output is a set of messages plus the updated history. Evaluation (Eval) naturally benefits precisely because the eval runner can play the role of a second host, injecting in-memory storage and stub dependencies so the same loop code runs in a controlled world, avoiding forking a second implementation just for testing.

Principle Three: Turn Errors into Results

Reliability lens: Does my trace faithfully reflect errors?

How you handle errors is critical to system reliability. The author found the best approach is to turn errors into outcomes. Unhandled exceptions short-circuit the turn's execution, potentially leaving conversation history in an inconsistent state, breaking subsequent message processing, and leaving an incomplete trace.

Take context overflow as an example: in the Anthropic API it manifests as a 400 error. If you let it surface as an error, the agent allows more messages into the conversation, each failing with no trace recorded. A better approach is to turn the 400 into a conversation_full result, letting the host declare the conversation full and reject new messages while the trace records it correctly.

The author handles errors at three levels: tools run in the executor and settle everything, the model client normalizes everything, and the loop catches the remainder as internal failures and logs them. Fail-fast is reserved only for trace-misuse guards and startup-time configuration validation.

Make the Trace a Lifecycle Guard

An important defensive pattern is to enforce the invariant: an opened span must be closed at the trace layer. When the host calls trace.end() at the end of a turn, any span still open is a lifecycle violation. In development this throws an exception; in production it becomes the reserved state undetermined—neither error nor missing, because the step never reported failure, while a missing span looks like a step that never ran.

The semantics of span and trace come from the distributed tracing model: one turn is a trace, each observable step within a trace is a span, and spans must be opened/closed in pairs to form a directed tree. The reliability significance is that "a span that wasn't closed" is itself a signal—it means some code path reported neither success nor failure, and the program may have abnormally jumped out of scope midway. If you simply record this unclosed state as missing, external observers will mistakenly think the step never ran and conclude that "the flow ended early normally." Marking it instead as the distinct undetermined state is equivalent to declaring at the data level that "there is something unknown here." This also explains why the guard must be environment-specific: throwing an exception directly in development exposes bugs as early as possible, while in production it becomes a reserved state that neither interrupts user requests nor lets observability lie.

Keep the Failure Alphabet Closed

A common trap is defining a few error types and wrapping everything else in a generic internal. If 80% of errors fall into the generic bucket, the damage to observability is no less than an incomplete trace. The author recommends ensuring the failure set is closed, and paying special attention to what can be pulled out of the internal bucket into its own category. In TypeScript, response satisfies never can ensure a new provider stop reason fails at build time.

The closed failure alphabet borrows from the "closed union" idea in type systems: if an error classification set can be exhaustively enumerated, callers can be forced by the compiler to handle every case; if a catch-all bucket like internal exists, any new error will be quietly swallowed into the bucket, distorting monitoring alerts and statistical baselines, resulting in a situation where "80% of traffic is internal and the root cause can't be located." The practical approach is: first enumerate all known provider stop reasons and network failure modes, then use exhaustiveness checks like response satisfies never to ensure that if an upstream adds an uncovered stop reason, TypeScript fails at build time rather than quietly falling into the catch-all at runtime. The cost is that provider upgrades may require adding types; the benefit is that the error distribution remains observable and can be handled in a targeted way.

Principle Four: History Is Atomic

Reliability lens: Does a failure pollute the conversation history sent to the model?

Every model call needs to send the full conversation history, and keeping that history clean and consistent is trickier than it sounds. Three typical pollution scenarios: the model hits max_tokens mid-tool-call, and a truncated tool_use enters history with no tool_result response; the transport fails after the customer message has been appended, and a retry appends it again; the model refuses and returns empty content that is nonetheless stored as a reply.

The solution is to keep history atomic: a turn either commits all its messages or commits none. A failed terminal state returns the history as-is, and the trace records the discarded messages. That way a failed turn neither poisons the conversation nor disappears.

Atomicity Cannot Undo Side Effects Already Executed

The author encountered a special problem in Kept: the model requests a tool call in the last round the budget allows, the tool call itself fits the budget, but the loop still needs an extra round for the model to process the tool response. The solution is to check the round limit before triggering the tool call rather than after. Context budgets are more complex, and pre-checks are at best estimates. Kept's approach is to detect overflow after the fact and convert it to conversation_full, with the real lever upstream: limiting the growth of a single round via result size and retrieval top-k.

Principle Five: Assert Every Transition in Three Ways

Reliability lens: Do my tests check what the agent did, or only what it said?

Since the state machine is built before the loop, every table cell is a test fixture. No matter how the tests are organized, the key is to ensure they assert at three levels: the outcome (the customer response cites Section 3.1), the content persisted in conversation history, and the shape of the trace and its expected content. This is a good rehearsal for evaluation.

Provider Contract Test Suite

Model calls are the core of an agent, and the model client is a sensitive component that deserves special attention. The author created a provider-agnostic interface for the LLM client and gave it a shared contract suite that every provider implementation must pass. The suite covers every way a call can end: a reply, parallel tool calls, max tokens, refusal, unknown stop reason, overflow mid-request and mid-generation, timeout, 500 error, network failure. When adding a new provider, the interface enforced by the same suite saved a lot of time chasing inconsistent provider behavior—it caught two drift bugs in Kept before conversations even started.

The Special Status of Tools

The author deliberately leaves tools out of this article's discussion, even though tool design and how agents interact with them are the biggest sources of unreliable behavior in agent systems. Tools should run in the executor, settling each call into a result the loop can hand to the model, no matter what happens internally. The author previews a dedicated article on tools that will cover why tool results have three states rather than two, who decides whether the model can retry, and why every result has one face for the model and another for the trace.

Conclusion

When agent products move from demos to production environments where a single wrong action can cost real money, understanding what the agent does in unplanned situations becomes more important than ever. We are no longer the people who write the code, and sometimes we don't even design the architecture or approve PRs. But we are still the ones who must take responsibility for the agent's behavior. When an agent misbehaves in production, engineering intuition remains the most valuable resource. The five principles in this article can serve both as a briefing for coding agents and as a checklist for reviewing what they build.

One-line Summary

Putting reliability first, five core design principles for building a production-grade AI Agent Loop.

Paragraph Summary

Drawing on the experience of building Kept, an open-source e-commerce after-sales agent, the article proposes five principles for keeping an Agent Loop stable in production: draw the state machine before writing code, covering all edge cases with terminal states, a transition table, and preconditions; separate the loop runner from the host, delegating invariants like conversation locks, configuration, persistence, and trace export to the host; turn errors into structured results instead of throwing exceptions, and safeguard observability with a closed failure alphabet and trace lifecycle guards; keep conversation history atomic, with a turn either fully committed or fully uncommitted; and test with assertions at three levels—outcome, persisted history, and trace shape—while building a shared contract suite for LLM providers. These principles also serve as a checklist for reviewing what an agent build has produced.