All Nine of My AI Voice Agents Were a Single Prompt—Until I Hit These Five Pitfalls

A developer manages nine client voice agents—HVAC, roofing, dental, water treatment, two law firms, credit repair, debt settlement, and an appointment setter. All of them use the same architecture: one giant prompt, no state machine, no flow diagram. Prompt lengths range from 2,400 to 4,400 words, with each agent carrying three or four tools.

This sounds like an anti-pattern, but the author insists it is "mostly right." He recently read all nine configuration files side by side and discovered a fact he had been reluctant to admit: most of the prompts are already state machines—they just write the states as prose.

This article is about where the line between a single prompt and explicit state lies, and what failure looks like when you end up on the wrong side of it.

Three Agent Structures, Explained in Developer Terms

The platform the author uses to build voice agents, Retell, offers three structural approaches.

Single prompt: one big block of instructions covering identity, scope, call steps, knowledge, escalation rules, and tool usage. The model reads all of it every turn and decides what to do next. The author likens it to "a big function with no explicit control flow, where the model is the interpreter."

Multi-prompt: the agent is split into multiple states, each with its own short prompt and tool list, plus transition rules between states. This is a state machine, where the body of each state is still natural language, but you define the edges.

Conversational flow: a visual graph. Each node is a step—say something, ask a question, call a function, branch on a condition, transfer. The call follows the path you drew. This is the closest thing on the platform to deterministic code, at the cost of rigidity.

According to the article, of the nine agents, the shortest prompt is about 2,400 words, and the longest is the personal injury lawyer intake line, at about 4,400 words.

State Machine is a computational model in software engineering consisting of a finite set of "states" and "transition rules" between them. The system is in exactly one definite state at any moment, and only transitions to another state when specific conditions (events or inputs) are met. In the context of voice agents, multi-prompt and conversational flow are essentially different manifestations of a state machine: multi-prompt writes each state as a separate short prompt, with transitions explicitly defined by the developer; conversational flow draws states and transitions as a visual graph. In contrast, a single prompt has no explicit state variables or transition table—the model can only "infer" which step it should currently be on from natural language descriptions. This is precisely the root of all the inconsistency problems described later. The key to understanding state machines is the enforceability of states and the determinism of transitions—that is, "does the system actually know which state it is in right now" and "under what conditions can it leave that state."

My "Single" Prompt Actually Contains Seven States

The author shows the heading structure of that 4,400-word lawyer agent: role and goal, personality, context, instructions, identity rules (critical), communication rules, scheduling rules, emergency escalation rules, stages, objection handling, tools, example conversations.

Under "stages," there are seven numbered stages: greeting, case type confirmation, time eligibility, injury and urgency assessment, qualification determination, scheduling, post-scheduling.

"Seven numbered stages. That is a multi-prompt agent wearing a trench coat," the author writes. The difference is that the model sees all seven stages, all tools, and all rules every turn, and there is no mechanism enforcing that "when it thinks it is in stage three, it really is in stage three."

As long as the rules never contradict each other, this works surprisingly well. But that is exactly the problem.

Prompt Scoping borrows the concept of variable scope from programming languages and refers to how broadly a rule or instruction applies. In traditional code, a local variable declared inside a function does not leak into other functions; but in a single prompt, all text lives in the same flat context, and the model reads all of it every turn. Therefore, a rule written as "before any tool call" actually applies to all tools listed in the prompt, including tools the author added later that did not exist when the rule was originally written. This lack of scoping has two consequences: first, old rules can unexpectedly contaminate the behavior of new tools; second, when the same behavior is described in multiple places, the model cannot determine which one has higher priority. Multi-prompt structures naturally achieve scope isolation by splitting rules across states, while a single prompt can only simulate it through conventions like "qualifying by name."

One Rule Written in Three Places, and Then the Three Versions Fought

The author shares an HVAC agent bug, the one he learned the most from.

The agent has a getWeekAvailability tool that loads availability for the next two weeks. The plan was simple: call it silently at the start of the scheduling stage, then ask the caller which day works. The original prompt said in the stage description: "At the start of the stage, before asking any questions, call silently once and do not announce it."

But the tools section had a general rule: "Before any tool call, use a brief verbal transition: 'Let me check' or 'One moment.'"

And in the example conversation, the agent first asks "Which day works for you?" and the tool call happens after that question.

The same behavior was specified three times in three sections, and all three contradicted each other. The model could not know which one was canonical and which was stale. Like anyone handed conflicting documentation, it picks one version each time it calls the tool—and not necessarily the same one every time.

The fix required changing all three places at once: the stage description changed to call immediately without announcing; the transition rule narrowed to "before calling checkCalendar or bookCalendar," with an explicit line stating "do not use a verbal transition for getWeekAvailability"; and every example rewritten so the tool call comes first and the question comes after.

The lesson the author extracts from this bug is at the code level, not the prompt level: "In a single prompt there is no scope. A rule written for 'any tool' applies everywhere forever, including tools you add later. Every general rule is a global variable, and every example conversation is a second copy of the logic that will drift from the first."

Why Are Nine Agents Still Single Prompts?

Because the cost of splitting is real, and for most of these calls it is not yet worth paying.

A single prompt is the most natural-sounding of the three structures. Callers do not follow your stages. They give their address while you are asking a question, ask about pricing during the scheduling step, or answer two questions at once. A single prompt can handle this because nothing stops the model from jumping around. In a multi-prompt agent, every jump is a transition you must design; a caller who asks about pricing during the scheduling state may get a blank stare. In a conversational flow, an answer with no predefined edge is a dead end.

Single prompts are also the easiest to modify. The author edits agents several times a week based on real calls, and a flat prompt read from top to bottom beats a graph where changing one node quietly breaks paths after three other nodes.

Conversational Flow in voice agent platforms usually refers to a visual orchestration approach based on a directed graph: each node represents a step (say something, ask a question, call a function, branch on a condition, transfer to a human), and edges represent allowed paths. Its advantage is strong determinism—callers can only move along the edges the developer drew, so compliance scripts, exact wording, and fixed sequences can be strictly guaranteed. The cost is rigidity: any input with no predefined edge (for example, a caller suddenly asking about pricing during scheduling) becomes a dead end, requiring the developer to enumerate all branches in advance. This contrasts sharply with a single prompt, which can tolerate nonlinear caller behavior because the model is free to jump. Conversational flow is best suited to scenarios where "the script is non-negotiable" and "call steps are highly predictable," such as legal disclosures, identity verification, and qualification determination.

Where Does the Author Draw the Line Now?

After reading side by side, the author lists the split signals he actually uses:

When the script is non-negotiable. If a client requires every caller to hear a disclosure statement before any questions, or a sales lead has signed off on the exact qualification wording, that is a conversational flow. A single prompt will follow the script most of the time, but compliance requires all of the time.

When a call does several things in sequence. Verify identity, look up records, schedule or reschedule, collect information needed for a follow-up text—each is a task with its own tools and failure modes. This is where multi-prompt starts to pay off, because each state gets a short prompt that can be tested independently.

When the tool list grows. The author's agents currently carry three or four tools, and calendar-related tools are already the site of his only real ordering bug. Beyond that number, the model starts reaching for the wrong function at the wrong time. Scoping tools to states solves most of this.

When you keep writing "critical." One section heading in the lawyer prompt is literally "Identity Rules - Critical." Shouting in a prompt means a rule is competing for attention with other rules. One or two is fine; when you need many, the prompt is asking to be split.

When your stages are numbered. If you have already written stage one through stage seven, you have already designed a state machine. The only remaining question is whether you will let the platform enforce it.

Four Habits for Staying with a Single Prompt

Since most agents will stay in a single prompt for a while, the author summarizes the habits he formed after the HVAC bug:

1. Say each rule only once. If a behavior is described in the stage, the tool description, and the example, either change all three or delete two.
2. Scope general rules by name. Write "before checkCalendar or bookCalendar," not "before any tool." New tools should not accidentally inherit old rules.
3. Treat examples as tests that can fail. According to the author's experience, models rely on examples at least as much as on instructions, so a stale example is a bug, not documentation.
4. Diff before publishing. The author puts the previous version and the new version side by side precisely for this. Testing a real phone line is a separate problem—phone calls have no repro.

Single Source of Truth (SSOT) is a principle in software engineering and documentation management: any piece of data or logic should be defined in only one place, and other locations should reference it rather than copy it. What the author encountered in the HVAC bug is a textbook case of SSOT being violated—the same "when to call the tool" behavior was written three times in the stage description, the tools section, and the example conversation, and the three copies gradually drifted and contradicted each other. In code, a compiler or test can catch this inconsistency; but in a natural language prompt, no mechanism reports an error, and the model simply picks one version at random to execute. Following SSOT means: keep only one authoritative definition of each rule, and either have example conversations reference that rule directly or update them in sync when the rule changes—otherwise the example becomes "a second copy of the logic" and a hidden source of bugs.

Clients Never See the Architecture

The author points out that none of his clients know or care whether their receptionist is a prompt or a twenty-node graph. What they care about is whether the phone gets answered and whether the appointment gets booked. This is one of the reasons they built VoiceDash—a white-label portal where each client can see their own calls, transcripts, and analytics. When an agent migrates from one structure to another, the client's view stays the same.

Transcripts are also how the author discovers whether an agent has outgrown its structure: the same step being skipped, or the same stutter after a topic change, recurring across multiple calls. That is the signal to split, not word count.

A Reusable Framework for Deciding

The value of this article is not a binary conclusion of "single prompt good or state machine good," but that it reduces prompt engineering to old software engineering problems: scope, single source of truth, examples as tests, and diffs as code review. If you are building an LLM agent with a long prompt—voice or not—the author's question is worth answering: where do you draw the line between one prompt and explicit state, and what finally pushed you over it?

This article is compiled from a developer's public analysis of nine voice agent configurations, with platform capability descriptions and agent configuration data drawn from the original source.