Agentic AI Development in Practice: Evaluation and Error Analysis Are the Real Competitive Edge

From Buzzword to Practice: The Two Faces of Agentic AI

When the term "Agentic" was coined to describe an important and rapidly growing trend observed in building applications on top of large models, few anticipated that it would quickly evolve into a marketing label slapped onto nearly every product. This phenomenon directly fueled a sharp rise in the hype surrounding Agentic AI.

Yet if we set aside the flashy promotion, we find a more noteworthy fact: the number of genuinely valuable and practical Agentic AI applications is also growing rapidly—though not as explosively as the hype. This is the key to understanding this technological trend: the buzz is illusory, but the ability to deliver real results is concrete.

This article is compiled from the content of Andrew Ng's Agentic AI course, aiming to help developers cut through the hype and see clearly how to truly build valuable agentic applications.

What is Agentic AI? At the technical level, Agentic AI refers to AI systems capable of autonomous planning, decision-making, and action. Unlike traditional single-turn Q&A large models, an AI Agent can break down complex goals into multiple subtasks, and through repeated reasoning, tool use, and interaction with the environment—dynamically adjusting its strategy based on feedback—ultimately complete long-horizon tasks. Its core architecture typically consists of four elements: Perception, Planning, Execution, and Memory. Frameworks such as ReAct, AutoGPT, and LangGraph have evolved around this architecture.

The Technical Evolution Behind Agentic AI

Agentic AI did not emerge out of thin air; it is the product of both breakthroughs in Large Language Model (LLM) capabilities and the evolution of software engineering paradigms. The explosion of ChatGPT in late 2022 made the public aware of the conversational abilities of LLMs, but what truly propelled the formation of the Agentic concept was a series of key developments that emerged around 2023: OpenAI's release of GPT-4 with Function Calling support, the AutoGPT project igniting the developer community on GitHub, and the rapid adoption of frameworks like LangChain. Together, these events validated a possibility—that LLMs can not only "talk" but also "do."

From an academic perspective, the theoretical foundations of Agentic AI can be traced back to the concept of Agents in reinforcement learning and the BDI (Belief-Desire-Intention) model in cognitive science. The BDI model was first proposed by philosopher Michael Bratman in his 1987 work Intention, Plans, and Practical Reason, then formalized by Rao and Georgeff in 1991 into a computable agent architecture, becoming the core theoretical basis for early multi-agent systems (such as the JADE platform). In the BDI model, "Belief" represents the agent's cognitive model of the current state of the world, "Desire" represents the set of goals it wishes to achieve, and "Intention" is the committed plan the agent is actively executing—these three follow a strict priority logic: when newly perceived beliefs conflict with existing intentions, the agent must decide whether to abandon its current intention in pursuit of a more important desire. This three-tier structure bears a striking correspondence to the modern LLM Agent's "world knowledge in the system prompt → goal task description → current execution step," revealing the deep influence of cognitive science on AI engineering design.

It is worth noting that the "agent" in reinforcement learning (RL Agent) shares a significant conceptual lineage with modern Agentic AI—both center on the core loop of "perceive environment state → select action → receive feedback → update strategy." The difference is that RL Agents rely on numerical reward signals for policy optimization, whereas LLM-based Agents use language as a universal interface to accomplish reasoning and planning. The profound significance of this shift is that natural language has become a "universal medium for planning," dramatically lowering the engineering barrier of designing dedicated reward functions for complex tasks, making the construction of complex autonomous systems more accessible than ever. Developers no longer need to meticulously design numerical rewards for every discrete action; instead, they can describe task goals and success criteria directly in natural language and let the LLM autonomously decompose the execution path.

What Agentic Workflows Are Changing

From "Q&A" to "Autonomous Loop": The Essential Upgrade of the Workflow Model

Traditional LLM applications are essentially static "question-answer" pipelines: the user inputs a prompt, the model returns output, and the process ends. Agentic workflows, by contrast, are dynamic, multi-step closed-loop systems. The model is not only responsible for generating text but also actively invokes external tools such as search engines, code interpreters, and database queries, deciding its next action based on the tool's returned results. This "perceive-decide-act-feedback" loop endows the system with the ability to handle open-ended, long-tail tasks—and precisely because of this, its behavioral uncertainty and debugging difficulty are far greater than those of traditional pipelines.

The underlying operating logic of this workflow model is best exemplified by the ReAct (Reasoning + Acting) framework. ReAct was proposed by Yao et al. from Google Research in 2022 (formally published at the ICLR 2023 conference), and experiments demonstrated that it significantly outperforms pure-reasoning or pure-action modes on multi-hop QA and interactive decision-making tasks—on benchmarks such as HotpotQA, ReAct improved accuracy by about 10% over pure chain-of-thought reasoning while substantially reducing hallucination errors. Its core idea is to have the LLM alternately generate two types of output—"Thought" and "Action": first perform reasoning analysis, then decide which tool to invoke and what parameters to pass, obtain the tool's returned result (Observation), and continue reasoning, looping until the task is complete. This alternating iteration of the "Thought-Action-Observation" triplet makes the model's reasoning process transparent and traceable, and provides a natural structured log for subsequent error analysis. Notably, ReAct's success also objectively propelled "Chain-of-Thought" from a mere reasoning technique into an executable action planning tool, a cognitive shift that has profoundly shaped the direction of the entire Agentic engineering practice.

In modern implementations, tool use (Tool Use/Function Calling) has been standardized into a unified JSON Schema interface: developers pre-define a tool's name, description, and parameter schema; the LLM automatically selects which tool to invoke during reasoning and generates a structured invocation request; an external runtime then executes it and feeds the result back into the context. The profound significance of JSON Schema standardization lies in the fact that the same business logic can be seamlessly connected to different underlying models, dramatically reducing migration costs and vendor lock-in risks; at the same time, the structured invocation format makes parameter validation and error capture of tool calls automatable. OpenAI's Assistants API, Anthropic's Tool Use, and Google's Function Calling all follow similar designs, while the Model Context Protocol (MCP) launched by Anthropic in 2024 goes a step further, attempting to standardize tool interfaces into an open, cross-vendor protocol, pushing the entire ecosystem toward interoperability. MCP's design philosophy bears a striking resemblance to the historical transition in early web services where the REST protocol gradually supplanted SOAP—the former won out with its concise, stateless interface design, while the latter was abandoned by the market due to excessive complexity. Whether MCP can become the "REST moment" of the Agentic tool ecosystem is worth continued attention.

Multi-Agent Architecture: The Evolutionary Direction of Single Agents

As the complexity of Agentic applications increases, the industry is evolving from single-agent architectures toward multi-agent systems. A single-agent architecture centralizes all planning, tool-calling, and execution logic in one LLM instance, with the advantages of simple implementation and coherent context, but the drawbacks that the context window can easily be exceeded when facing long-horizon complex tasks, and that different subtasks are hard to process in parallel.

Multi-agent architecture introduces the "Orchestrator-Worker" pattern: a top-level planning agent is responsible for task decomposition and scheduling, multiple specialized sub-agents process subtasks in their respective domains in parallel, and the orchestrator finally aggregates the results. This pattern has a natural analogy to the microservices architecture in software engineering—decomposing a monolithic application into single-responsibility, independently scalable service units in exchange for the overall system's flexibility and maintainability. Frameworks such as LangGraph, AutoGen (open-sourced by Microsoft), and CrewAI all provide native support for multi-agent orchestration.

Among them, the AutoGen framework introduces the quite innovative "Conversational Programming" paradigm, abstracting inter-agent interaction into a programmable structured conversation flow. Under this paradigm, developers no longer need to manually write complex state machines or task scheduling logic; instead, they implicitly describe collaboration logic by defining agent roles (such as "Planner," "Critic," "Executor"), permission boundaries, and conversation termination conditions. This design allows complex multi-agent collaboration to be defined in a manner close to natural language, greatly lowering the development barrier for multi-agent systems, while also supporting Human-in-the-loop intervention at any node in the conversation flow for approval or correction. LangGraph, meanwhile, borrows the concept of a Directed Acyclic Graph (DAG), expressing the workflow explicitly as a graph structure of nodes (state processing functions) and edges (conditional transition logic). Compared to linear chained invocation, the graph structure allows workflows to contain branches, loops, and parallel paths, providing fine-grained control while also significantly improving the visualization and debuggability of complex processes—developers can directly "see" the agent's execution path rather than inferring it in reverse from logs.

It is worth noting that multi-agent systems also bring new engineering challenges: state synchronization and consistency assurance is among the thorniest—when multiple sub-agents modify shared state in parallel, mechanisms similar to database transaction optimistic locking or Event Sourcing must be introduced to prevent data races. Event Sourcing is a classic pattern in the distributed systems field, whose core idea is to record all state changes as an immutable sequence of events (Event Log) rather than directly overwriting the current state snapshot—this means any historical state of the system can be precisely reconstructed by replaying the event sequence, achieving both complete audit traceability and natural support for checkpoint retries after failure. This property is especially valuable in Agentic systems that require full reasoning-chain replay, particularly when task execution spans hours or even days.

Moreover, Cascading Failure is another easily underestimated systemic risk in multi-agent systems: when an upstream sub-agent produces erroneous output, if effective validation and fault tolerance mechanisms are lacking, the error propagates down the call chain and is progressively amplified, ultimately causing the overall task to fail in ways that are difficult to diagnose. This phenomenon is highly similar to the "avalanche effect" in microservices architecture, and engineering practice often uses patterns such as the Circuit Breaker—automatically interrupting calls to a node when a sub-agent's failure rate exceeds a threshold—and the Bulkhead—allocating independent resource pools to different types of tasks to prevent a single task type from exhausting system resources—to control the spread of faults. This is precisely the fundamental reason why a layered Evals system is indispensable in multi-agent scenarios: only by precisely tracking the output quality of each sub-agent can we provide early warning before cascading failures occur.

Real-World Scenarios Already in Production

Today, Agentic workflows have been widely applied across many real-world scenarios. These are not mere theory but applications actively generating tangible business value:

• Customer Support Agents: automatically handling user inquiries, understanding context, and providing precise responses;
• Deep Research Assistants: assisting in writing research reports with profound insights;
• Legal Document Processing: parsing complex, thorny legal documents and extracting key information;
• Medical Diagnostic Assistance: analyzing patient input information and offering possible diagnostic suggestions.

Andrew Ng specifically noted that among the many teams he leads, a considerable portion of projects would simply be impossible to accomplish without the support of Agentic workflows. The weight of this statement lies in the fact that AI Agents are not a "toy" that merely adds icing on the cake, but a key leap that turns certain complex tasks from "impossible" to "possible."

Why Mastering Agentic Development Is the Most Important AI Skill Right Now

Precisely because Agentic workflows can unlock entirely new application possibilities, mastering how to build them has become one of the most important and valuable skills in the AI field today. Whether for landing a job or independently developing stunning software products, this ability can open the door to broad opportunities for you.

The Dividing Line Between Experts and Novices: Evals and Error Analysis

A Severely Underestimated Core Competency

The most central and enlightening point in the course is this: Andrew Ng observed that the biggest difference between developers who are truly skilled at building Agentic workflows and those who achieve mediocre results is not their proficiency with a particular framework or tool, but rather whether they can drive a disciplined development process.

The core of this process focuses on two things:

• Evals (Evaluation): systematically measuring the output quality of the agent;
• Error Analysis: deeply dissecting the reasons for agent failures and pinpointing the root cause of problems.

This viewpoint is quite subversive. Many developers tend to be keen on trying various new frameworks and piling up complex prompts, yet neglect the "boring" but success-determining key link of rigorously evaluating the system.

Understanding Evals in Depth: The "Unit Tests" of AI Systems

In AI engineering practice, Evals (Evaluations) refers to a testing framework used to systematically measure the output quality of a model or agent. Analogous to unit testing and integration testing in software engineering, Evals provide a quantifiable quality baseline for AI systems. A well-developed Evals system typically includes: a Golden Dataset, automated scoring metrics (such as accuracy, recall, and consistency), and a human review process.

Building the Golden Dataset is the most time-consuming and most critical link in the Evals system. It is widely believed in the industry that a high-quality test set should cover three categories of samples: typical scenarios (Happy Path, covering 80% of routine use cases), edge cases (testing the system's stability under extreme inputs), and adversarial cases (simulating anomalous inputs deliberately constructed by users). Among these, the construction of adversarial samples is especially subtle—it includes not only language-level challenges such as semantic ambiguity and syntactic malformation, but should also cover attack surfaces unique to Agentic systems, such as cross-step state manipulation and forging of tool return values—for example, injecting misleading information into content returned by a search tool to observe whether the agent can identify and resist "Prompt Injection" attacks. The Golden Dataset needs continuous maintenance and updating as the business evolves; otherwise, the test set itself will gradually lose representativeness and become a bottleneck to system progress rather than an aid.

The LLM-as-Judge approach (using an LLM to evaluate LLM output) that has emerged in recent years can substantially reduce manual annotation costs, but one must beware of the "systematic bias resonance" problem between the evaluating model and the evaluated model—that is, the two models may share the same training data bias, leading to inflated evaluation results. In practice, common mitigation strategies include: using a model from a different vendor or a different architecture than the evaluated model as the Judge (e.g., using Claude to evaluate GPT-4o's output), explicitly listing positive and negative examples for each scoring dimension in the evaluation prompt to reduce scoring ambiguity, and periodically validating the scoring consistency of the LLM-as-Judge with human sampling—typically requiring the Cohen's Kappa coefficient between the Judge and human scores to exceed 0.7 before it can be trusted (the Kappa coefficient is a statistic measuring the agreement between two raters; 0 indicates random agreement, 1 indicates perfect agreement, and 0.7 is generally accepted by academia as the threshold for "strong agreement"). For Agentic systems, Evals are especially critical—any deviation in an intermediate step of multi-step reasoning can cause the final output to spiral out of control, and without an evaluation system, there is no way to pinpoint which step the problem occurred in.

From an industry practice perspective, the complexity of evaluation far exceeds that of single-turn dialogue, and the consensus gradually forming in the industry is a "layered evaluation" strategy:

• Component-level evaluation: the accuracy of individual tool calls and the validity of parameter generation;
• Process-level evaluation: the completeness of the reasoning chain and whether there are redundant loops or ineffective tool calls;
• End-to-end evaluation: the comprehensive trade-off among task completion rate, user satisfaction, and system latency.

These three levels of evaluation complement each other and are all indispensable: component-level evaluation helps quickly locate single-point failures, process-level evaluation reveals systematic flaws in the agent's reasoning strategy, and end-to-end evaluation ensures that local optimization does not come at the cost of the overall user experience—this is highly consistent with the design philosophy of the three-tier structure of unit testing, integration testing, and E2E testing in the software quality assurance system.

Startups such as Braintrust and LangSmith (under LangChain) specialize in providing Evals-as-a-Service platforms for Agentic systems, whose core value lies in providing visual replay of execution traces—developers can replay the agent's reasoning process step by step, much like single-stepping through code during debugging, precisely locating failure nodes in a multi-step reasoning chain. Such platforms typically also offer regression testing capabilities: after each prompt update or underlying model switch, they automatically run the complete evaluation suite on the Golden Dataset to prevent the hidden regression of "the new feature fixed A but quietly broke B." The rise of such tools is itself an important signal: the Observability of Agentic systems is becoming an independent engineering subfield, just as the microservices era gave rise to distributed tracing tools like Datadog and Jaeger—its strategic importance is no less than that of model capabilities themselves.

Why the Evaluation System Determines Project Success or Failure

Agentic workflows typically involve multi-step reasoning, tool calls, and external interactions, and their behavior carries a degree of uncertainty. In such a complex system, without a reliable evaluation mechanism, developers simply cannot judge whether a given change made the system better or worse.

Error Analysis, meanwhile, is the foundation that makes development iteration controllable. This methodology was systematically promoted by Andrew Ng in his classic ML engineering course, and its core steps are: randomly sample from failure cases, manually annotate the error types (such as information retrieval failure, reasoning chain breakage, tool call error, etc.), tally the frequency and impact of each error type, and thereby determine optimization priorities. This methodology has over twenty years of accumulated practice in the machine learning field—from early confusion matrix analysis to modern Slice-based Evaluation, the core logic has always been "You can't improve what you can't measure." Slice-based evaluation refers to splitting the test set into subsets along specific dimensions (such as user region, task type, input length, industry domain) and computing performance metrics for each subset separately, thereby discovering local performance regressions masked by overall metrics—for example, an agent with an overall accuracy of 85% may only achieve 60% on the "involving numerical computation" slice, and this local defect could easily be overlooked without slice-based evaluation. In Agentic systems, slice-based evaluation is especially important, because different types of tasks often correspond to entirely different failure modes: the main failure point of information retrieval tasks lies in wrong tool selection, while multi-step reasoning tasks are more prone to cumulative deviations in the conclusions of intermediate steps.

In Agentic systems, error analysis faces greater challenges—a single task failure may involve a dozen steps of reasoning chain, requiring step-by-step backtracking to locate the root cause. This requires developers to have complete Trace Logging capability: the input and output of each step, the parameters and return values of tool calls, and the intermediate reasoning process of the model all need to be fully recorded for later reproduction and analysis. An ideal Trace log should have three characteristics: completeness (not missing any intermediate state), structure (facilitating programmatic querying and statistics), and low overhead (not significantly increasing system latency). At the same time, developers need the analytical ability to classify and abstract complex failure modes—for example, categorizing "reasoning deviation caused by irrelevant search results" and "task interruption caused by tool call timeout" into different error categories and formulating optimization strategies separately; the former requires improving the query generation logic of the retrieval tool, while the latter requires introducing timeout retry mechanisms and fallback strategies.

Only by systematically analyzing at which links and in what ways the AI Agent goes wrong can developers optimize in a targeted manner, rather than blindly adjusting on gut feeling. This is precisely the essence of a "disciplined development process"—driving iteration with data and analysis rather than relying on intuitive trial and error.

Three Takeaways for Developers

For developers hoping to enter or deepen their work in the Agentic AI field, the signal conveyed by this course is very clear:

First, don't let the hype throw you off pace. The value of agents is real, but it needs to be viewed rationally. Rather than chasing every new concept, focus on building LLM applications that truly solve problems.

Second, put your effort into evaluation and error analysis. Compared to framework selection, establishing a reliable Evals evaluation system is the real threshold that separates skill levels. This is a core investment that is easily overlooked but yields extremely high returns—it makes every iteration of yours traceable and its effectiveness measurable.

Third, seize the opportunity of the era. As Andrew Ng emphasized, the ability to build Agentic workflows is one of the most valuable skills in the AI field today. Whether for career development or personal projects, now is a great time to jump in and learn.

Conclusion

The wave of Agentic AI hype will eventually subside, but the ability to truly build valuable applications will accumulate over the long term. Rather than dwelling on the endless stream of marketing jargon, it is better to steadily establish a development methodology centered on evaluation and error analysis. When you can drive the iterative optimization of an AI Agent with a rigorous process—with a test set, error attribution, and quantitative metrics—you will already be standing at the truly correct starting point of this field.