90% of Agents Fail at Launch: A Four-Layer Architecture Breakdown from Conversation to Execution

The pace of concept evolution in AI feels like a sprint — from prompt engineering to agentic loops, and now to Ontology and FDE. On the surface it looks like technical iteration, but the core has always pointed to the same thing: getting AI to actually do real work. Over the past two years, the industry competed on which model scored higher. Now the competitive standard is quietly shifting — who can actually get things done.

In a video on Bilibili, creator Liang Laoshi used a real-world cross-border e-commerce return and refund scenario to break down agent deployment to an actionable level of detail. His core argument: the gap between "can hold a conversation" and "can get things done" isn't model intelligence — it's an entire engineering stack. Most teams only build the first layer, which is why their demos dazzle and their production deployments crash.

AI Is Shifting from "Interface" to "Workforce"

Three seemingly unrelated industry developments all point in the same direction.

Moving upward into enterprises: South Korean telecom KT launched an enterprise agent platform called Agentic On, connecting AI to existing company data and systems to handle multi-step automation tasks like document review, approval routing, and system data entry. Moving into hardware closer to us: Meta announced plans to integrate its personal agent Muse into its AR glasses — users can invoke it with a single phrase without touching their phone, and the agent can execute tasks based on what the user is currently looking at. Moving down to the compute layer: Bernstein noted that agents expand a single Q&A exchange into a full pipeline of "plan → retrieve → call tools → execute → verify," simultaneously driving GPU inference load, CPU demand, memory caching, databases, and storage. Anthropic's seven-year, $11.6 billion agreement with a compute vendor explicitly targeting CPU workloads is a signal that compute demand is spreading well beyond GPUs.

Put the three together and the conclusion is clear: AI is shifting in form from "interface" to "workforce."

The Fundamental Difference Between Conversational AI and Execution-Oriented Agents

The difference isn't in the model — it's in who bears responsibility for getting it done.

Conversational AI follows this loop: I ask, it answers, I decide what to do next. Responsibility stays with me; its output is just text. An execution-oriented agent follows a different loop: I hand it a task, it plans independently, calls tools, executes, checks results, retries on failure, and hands me back something already completed — responsibility transfers to the system.

The real process of an agent completing a task is a loop, not a single exchange: understand the goal → break down and plan → retrieve context → call tools → execute actions → check results → retry or escalate to a human if it fails. Of those seven steps, only the first one involves "conversation." The remaining six are pure execution. This also explains why agents simultaneously drive inference, caching, databases, and storage — they're cycling through this loop repeatedly, not generating a one-shot response.

The root of the difficulty: conversation can be fuzzy; execution must be precise. If a chat response is wrong, the user smiles and asks again — the cost is nearly zero. If an executed action is wrong, money has already been transferred, a ticket has already been closed, a database has already been modified — the cost is real money and broken trust. This is why Liang Laoshi emphasizes: 90% of the work in agent engineering isn't writing prompts. It's designing fallback paths for when things go wrong.

The First Step to Deployment: Filter Scenarios with Four Criteria

The starting point for deployment isn't choosing a model — it's choosing the right scenario. The video offers four screening criteria:

• High frequency: Hundreds or thousands of times per day, so the labor savings are actually visible;
• Multi-step: At least three steps spanning two or more systems — otherwise a script is sufficient;
• API-accessible: The target system has an interface and doesn't rely purely on manual keyboard input;
• Recoverable: If something goes wrong, it can be rolled back, or a human can catch it.

All four need to check out before it's worth investing.

Using cross-border e-commerce returns as an example: the user says just one sentence — "The shoes I bought last week gave me blisters after one wear, I want to return them and get a refund" — but for the agent to actually complete this task, it takes eight steps: understand the request → check the order system → check the logistics system → read the return policy → determine eligibility → call the refund API → update the ticket and customer record → notify the user. Only the first step involves conversation. The remaining seven are all execution.

A Deployable Agent Must Be Built in Four Layers

To actually run through all eight steps, a deployable agent needs at minimum four layers:

• Brain layer: Responsible for decomposing tasks and deciding next steps;
• Tool layer: Wraps enterprise systems into callable interfaces with clearly defined contracts;
• Context and state layer: Tickets are stateful, long-running tasks that require memory and a state machine — not a one-off conversation;
• Governance layer: Handles permissions, approvals, auditing, and circuit breakers.

Liang Laoshi observes that most people only build the first layer (the brain layer), which is why "the demo is stunning, and the production launch falls apart."

Three Iron Rules for the Tool Layer

The tool layer is the most underestimated layer, yet the most decisive for production success. Giving an agent a pair of hands requires following three iron rules:

First, keep interfaces narrow. Don't give it an all-purpose executor — give it a deterministic tool like "check order status," with explicit parameters, controllable outputs, and traceable errors. Second, make actions idempotent. Agents will retry, and retries must not result in duplicate refunds. Every financial action should carry an idempotency key so that repeated calls are safe. Third, enforce least-privilege permissions. Read operations can be open; write operations and financial operations must be limited by amount, scope, and frequency.

In short: the boundary of the tools is the ceiling of the agent's capabilities.

In the refund API implementation, the boundary is tightened by three layers of constraints: idempotency guarantees that retries don't result in duplicate refunds; a cap ensures no single transaction exceeds 300 yuan, with anything above that routed to manual approval; and the call parameters directly encode the rules — amounts above 300 or confidence below 85% escalate to a human. The critical point is that these constraints are not written into prompts hoping the model will "self-regulate" — they are enforced in code. This is the dividing line between a demo and a product.

Idempotency is a foundational concept in distributed systems engineering: executing the same operation once or multiple times leaves the system in exactly the same final state. In agent scenarios, due to network timeouts, LLM retries, or tool call failures, the same refund request may be sent multiple times. The standard practice for an idempotency key is to include a unique identifier (typically a UUID) in each request; the server caches the result after the first successful processing and returns the cached result for subsequent requests with the same key without re-executing. Payment systems like Alipay and Stripe enforce idempotency keys as mandatory parameters for financial APIs. In agent engineering, "make actions idempotent" is an iron rule because LLM-driven planners are inherently non-deterministic and retries are practically unavoidable at the architecture level. Without idempotency protection on financial interfaces, a simple network hiccup can directly turn into a duplicate charge or duplicate refund — a real financial loss.

State machines are formal models describing the states an object occupies throughout its lifecycle and the rules governing transitions between them. They consist of three elements: states, events, and transitions. Using a return ticket as an example, its states might progress through: submitted → eligibility review → awaiting customer shipment → logistics confirmation → refund processing → completed / rejected. Each state can only transition to another through specific legitimate events — for example, "rejected" cannot jump directly to "refund processing." Conversational AI doesn't need to track these states — when the user sends a message, the model just generates the next reply. An execution-oriented agent, however, must always know — across multi-step, time-spanning tasks — "where is this task right now," "which node did the last failure occur at," and "what are the valid next actions." This is exactly why the context and state layer exists. Without state management, an agent that fails at any intermediate step cannot correctly resume — it can only restart from scratch or get completely stuck.

Gating: Knowing When to Stop

The real question that determines success or failure isn't whether the agent can do something — it's whether it should do it on its own.

The video uses two dimensions — transaction amount and confidence score — to define a tiered decision rule: low amount plus high confidence allows automatic execution; low amount but medium confidence routes to another model or rule-based cross-validation; very low confidence triggers human takeover. And once the amount exceeds 300 yuan, cross-validation is required even with high confidence; above 2,000 yuan, human review is mandatory and financial sign-off is layered on top.

The conclusion is worth reflecting on: the maturity of automation isn't measured by how boldly it acts — it's measured by knowing when to stop.

The confidence score here is not the softmax probability from the model output — it's a custom composite business-layer score, typically derived by weighting multiple signals: the match quality of intent recognition, the completeness of key slot extraction (order number, amount, reason), the clarity of policy rule matching, and the consistency of handling similar historical cases. Setting confidence thresholds is fundamentally a risk-reward tradeoff: set it too high and large volumes of simple, auto-resolvable tickets get pushed to humans, limiting efficiency gains; set it too low and the model forces automatic execution on semantically ambiguous or edge cases, driving up error rates. In practice, the recommended approach is to collect real data in shadow mode, use ROC curve analysis to evaluate automation rates and error rates at different thresholds, and then determine the final threshold based on the business's acceptable cost of errors — rather than picking an arbitrary fixed number.

How to Evaluate After Launch: Don't Fixate on a Single Metric

Once an agent is live, evaluation cannot focus on a single metric. The video recommends tracking five dimensions:

1. End-to-end task success rate, not single-step accuracy — every one of the eight steps being individually correct doesn't mean the overall task was completed;
2. Human takeover rate — lower is better, but it should never reach zero; zero typically means gating has failed;
3. Cost per task, including tokens, tool calls, and human time;
4. Error cost — the financial loss caused by incorrect actions;
5. Average completion time.

It's also strongly recommended to run shadow mode before going live: the agent produces a plan that humans execute, and the differences between the two are compared before gradually increasing the agent's autonomy.

Agents Change Processes, Not Models

At a higher level, deploying an agent changes processes, not models. The human role shifts from executor to rule-setter and exception handler. Processes must preserve a "human entry point" — any critical action can be halted and taken over by a person. And responsibility must be defined upfront: who ultimately signs off on decisions the agent makes.

Liang Laoshi closes with an analogy: treat the agent like a newly hired employee. You need to give them a job description (boundary of responsibilities), an operations manual (tools), a probationary period (shadow mode), and a supervisor (gating). Missing any of the four, and it has no business going into production.

Compressed into five iron rules: define the boundary before talking about capability; integrate the systems before optimizing the dialogue; design for failure before designing for success; run shadow mode before going live; measure end-to-end before measuring individual steps.

Three sentences to take away the core message: conversational AI sells answers; agents sell outcomes. Outcomes can be measured and held accountable — that's both the opportunity and the barrier to entry. The dividing line in agent development isn't how boldly it automates — it's how well it knows when to stop.