Andrew Ng Teams Up with LangChain: Core Principles of LangGraph Agent Development

From LLMs to Agents: An Evolution in Workflow Paradigms

The AI education platform DeepLearning.AI has released a course titled AI Agents in LangGraph, co-created by Andrew Ng, LangChain co-founder and CEO Harrison Chase, and the co-founder and CEO of Tavily. Widely regarded by developers as an excellent introductory tutorial for agent development, this course systematically covers how to build AI Agents from scratch using the LangGraph framework.

In the course introduction, Andrew Ng reflected that just about a year ago, when the team created their first LLM framework course, building a reliably working agent example was still quite difficult. Today, the situation has fundamentally changed. According to Harrison Chase, the ability to deploy agent applications at scale is primarily due to two key improvements: first, the maturation of Function Calling capabilities, making tool invocation more predictable and stable; and second, the agent-adapted design of specialized tools, with search tools being the most prominent example.

Function Calling Background: Function calling is the foundational mechanism for modern LLMs to use tools. OpenAI officially introduced the Function Calling feature in GPT-3.5 and GPT-4 in June 2023, allowing models to output structured JSON-format instructions specifying function names and parameters instead of free-form text. The maturation of this capability significantly reduced the uncertainty of Agent tool invocation—early Agents often relied on complex prompt engineering and text parsing to drive tool usage, resulting in high error rates and maintenance difficulties. Subsequently, Anthropic's Claude, Google's Gemini, and other mainstream models also rolled out similar Tool Use mechanisms, gradually establishing an industry standard. Notably, function calling is essentially a form of "controlled output formatting": models are trained to preferentially generate structured responses conforming to JSON Schema constraints in specific contexts, rather than arbitrary text—this relies on Instruction Fine-tuning and Reinforcement Learning from Human Feedback (RLHF) to directionally shape the output distribution. After 2024, the industry further generalized this concept into "Tool Calling," supporting parallel triggering of multiple tools in a single inference pass, significantly improving Agent execution throughput for complex tasks and laying the technical foundation for parallelized orchestration of multi-step tasks.

This observation hits the core pain point of agent development—traditional tools were not designed for AI Agents. Take search engines as an example: when a regular user queries, multiple links are returned for manual browsing. But for an autonomously running Agent, what it truly needs is directly citable answers with source links in a predictable format.

Traditional search engines (like Google and Bing) are designed to provide human users with lists of relevant web pages—their output consists of link collections that require manual clicking, reading, and filtering, implicitly assuming that "humans will handle ambiguity and unstructured information." Agents, however, need programmatically consumable information: structured text summaries, explicit source citations, consistent output formats, and deep understanding of query intent. Agentic Search tools like Tavily are designed precisely for this need—their APIs directly return refined answer text and source lists rather than HTML pages. Technically, this design relies on a variant of Retrieval-Augmented Generation (RAG): first obtaining raw content through web crawling and indexing, then using LLMs to summarize and synthesize multi-source content, and finally outputting structured answers with verifiable source links for citation tracing.

It's worth adding that RAG technology itself has evolved from "Naive RAG" to "Advanced RAG" to "Modular RAG." Naive RAG performs simple retrieval + concatenation, while Advanced RAG introduces optimization steps like query rewriting, hybrid retrieval (dense vectors + sparse BM25), and reranking. The design philosophy of Agentic Search tools is closer to Modular RAG—decomposing retrieval, filtering, and summarization into independently optimizable components, with an Agent driving the entire pipeline, thereby simultaneously surpassing static knowledge base retrieval in real-time performance, accuracy, and explainability. Similar tools include Exa (formerly Metaphor) and the Perplexity API, which together form a crucial part of the next-generation Agent infrastructure, bridging the gap between LLM reasoning capabilities and real-time internet information.

What Is an Agentic Workflow?

To help developers intuitively understand Agents, Andrew Ng used a vivid analogy to explain "Agentic Workflow." Imagine three people collaborating on a research paper: one person plans and outlines; another researches, retrieves materials, and organizes documents; a third writes the initial draft; then someone reads through the entire piece, offers revision suggestions, and sends it back to the appropriate role for revisions or additional research—iterating in this loop until the final product is delivered.

This stands in stark contrast to how most people use LLMs today. Most people writing with large models give a single prompt and have the model generate the entire piece one-shot from start to finish. An agentic workflow, through iterative refinement, can produce results of much higher quality. Andrew Ng offered an elegant analogy: even he himself, if required to write straight through without backtracking or editing, wouldn't produce good work. If human writing requires repeated revision, how much more so for LLMs.

From a cognitive science perspective, this iterative workflow closely aligns with the human "Dual Process Theory": fast intuitive generation (System 1) works in tandem with slow critical review (System 2). In Agent architecture, initial draft generation corresponds to fast single-pass inference, while reflection and revision simulate slow thinking—using multiple LLM calls to compensate for the limitations of single-pass reasoning. Research shows that even using the same base model, introducing iterative reflection mechanisms can improve accuracy by 20%-40% on tasks like code generation and mathematical reasoning.

This idea is highly consistent with the widely discussed "Test-Time Compute Scaling" theory: investing more computational resources during inference (i.e., more LLM calls and iterative steps) is often more effective at improving performance on complex tasks than simply scaling up model parameters. OpenAI's o1/o3 series models internalize this idea as a model training objective, while Agent frameworks provide a more flexible external implementation path at the application layer, enabling developers to benefit without waiting for new models.

In practical terms, this means separately prompting the LLM to write outlines, draft initial versions, revise drafts, execute searches, and so on—decomposing complex tasks into iterable steps.

Five Core Design Patterns for Agents

In the course, Andrew Ng systematically outlined the key design patterns for agentic workflows, which form the foundation for understanding modern Agent architecture:

1. Planning

This means thinking about which steps need to be executed—like outlining a paper before deciding on subsequent actions. Planning capability determines whether an Agent can decompose vague goals into executable subtasks. In Agent engineering, planning typically follows two implementation paths: static planning (like Tree-of-Thoughts), which generates a complete plan before execution; and dynamic planning (like ReAct), which adjusts subsequent plans in real-time based on observations after each action. The former suits well-structured tasks, while the latter better handles open environments with high uncertainty.

The choice between static and dynamic planning is fundamentally a trade-off in "commitment timing." Static planning commits to a global path before execution, suitable for scenarios with clear task boundaries and high environmental certainty, but if any intermediate result deviates from expectations, the entire plan may need to be scrapped, lacking fault tolerance. Dynamic planning follows the "Least Commitment Principle"—making decisions only for the current step and deferring subsequent path determination until actual observational feedback is received, trading stronger adaptability for increased reasoning overhead (each step requires a full LLM inference). For production Agent systems, a common compromise is hierarchical planning: first generating a coarse-grained high-level plan (static), then performing fine-grained dynamic adjustments during the execution of each high-level step, balancing planning efficiency with execution flexibility.

2. Tool Use

Knowing which tools are available and how to invoke them—such as the search tools mentioned earlier. Tool use capability primarily relies on the LLM's own function calling features. From an engineering practice perspective, tool set design is itself a discipline: tools that are too coarse-grained limit Agent flexibility, while tools that are too fine-grained increase planning complexity and invocation overhead. Mature Agent systems typically follow the "orthogonality principle"—each tool handles a single, well-defined responsibility, with complex functionality achieved through compositional invocation.

The quality of Tool Descriptions is often underestimated by beginners, but it has a decisive impact on Agent performance. When an LLM decides whether to invoke a tool and how to populate its parameters, it primarily relies on the tool's natural language description (i.e., the function docstring or the description field in the schema) for reasoning. Research shows that clear, precise tool descriptions can improve tool selection accuracy by 15%-30%. Additionally, when the number of tools exceeds a certain threshold (generally considered to be 20-30), including all tool definitions in a single prompt causes a "tool retrieval noise" problem—the model's accuracy degrades when selecting among a massive set of tools. To address this, the industry has developed Tool Retrieval techniques: first using semantic search to dynamically retrieve the most relevant tools from a tool library, then injecting them into the current prompt, thereby balancing tool set scale with invocation accuracy.

3. Reflection

This refers to iterative improvement of results, often requiring multiple LLMs to critique each other and offer valuable suggestions, thereby driving the editing loop. This is the architectural embodiment of the "iterative refinement" philosophy. Reflection mechanisms are typically implemented through two technical approaches: Self-Critique, where the same model reviews its own output using critical prompts; or Adversarial Verification, where a dedicated "Critic" model independently evaluates the output of a "Generator" model—a design philosophically akin to Generative Adversarial Networks (GANs).

It should be noted that self-critique and adversarial verification each have their applicable boundaries. The limitation of self-critique lies in "blind spot consistency"—if a model makes certain systematic errors during generation (such as misunderstanding domain knowledge), its critic role will often make the same errors, causing the reflection loop to become "self-consistent" on the wrong track. To break this "self-enclosed" cycle, introducing external benchmarks (such as code execution results, unit test pass rates, fact-checking APIs) as objective feedback signals is essential. This is precisely the key design philosophy behind code Agents like AlphaCodium—using the deterministic output of compilers and test suites as anchors for reflection, rather than relying on the model's own subjective assessment, giving the reflection mechanism an objectively verifiable foundation.

4. Multi-Agent Communication

Think of each Agent as a team member playing a specific role, each LLM equipped with a unique prompt, fulfilling a distinct responsibility in the overall workflow—just like the planner, researcher, writer, and reviewer in collaborative paper writing.

Multi-Agent Systems (MAS) are a classic research area in artificial intelligence, with theoretical roots tracing back to distributed AI research in the 1980s. Multi-agent collaboration in the modern LLM era builds on this foundation by incorporating ideas from Role Playing and Mixture of Experts: different Agents are given different system prompts and tool sets, collaboratively handling complex tasks that exceed the context window or capability boundaries of a single LLM. The core challenge in multi-agent architectures is communication protocol design—how Agents pass state, negotiate disagreements, and handle conflicts directly determines the overall system reliability.

From a system design perspective, there are three main communication topology patterns for multi-agent systems: Centralized Orchestration (where an "orchestrator" Agent dispatches all "executor" Agents), Decentralized Peer-to-Peer communication (where Agents interact directly with each other), and Hierarchical hybrid structures (tree-like multi-level orchestration). Centralized orchestration is easy to debug and control but creates single-point bottlenecks; decentralized communication is more flexible but state synchronization is complex; hierarchical structures strike a balance between scalability and controllability. LangGraph natively supports Subgraph nesting, making it naturally suited for building hierarchical multi-agent systems, while AutoGen leans toward a decentralized, conversation-driven model—these two paradigms represent the two mainstream philosophies in current multi-agent engineering. Frameworks like AutoGen and CrewAI, along with LangGraph, constitute the main competitive landscape in this space, each with different emphases on communication models and orchestration strategies.

5. Memory

This means tracking progress and results across multiple steps. Memory mechanisms allow Agents to remember previous states and outputs, which is critical for maintaining coherence across long task chains. Agent memory systems are typically divided into four layers: Working Memory (information within the current context window), Episodic Memory (stored historical conversations and task records), Semantic Memory (knowledge base stored via vector databases), and Procedural Memory (skills solidified through fine-tuning or prompt caching). Different types of memory require different storage media and retrieval strategies, making this one of the most complex components in Agent engineering.

These four memory layers don't operate in isolation—there exists a dynamic "Memory Consolidation" mechanism: high-value information from working memory can be compressed into summaries and transferred to episodic memory; frequently retrieved episodic memories can be distilled into structured knowledge in semantic memory; and efficient problem-solving patterns used repeatedly can be solidified as procedural memory (through few-shot example caching or fine-tuning). This process has a profound analogy to the mechanism by which the hippocampus consolidates short-term memory into long-term memory in cognitive neuroscience. In engineering implementation, LangGraph's persistence mechanism (Checkpointer) primarily covers the working memory and episodic memory layers, while semantic memory and procedural memory management typically requires integration with vector databases (such as Pinecone, Weaviate, Qdrant) and external knowledge management systems to form a complete memory infrastructure stack.

You may not have noticed, but Andrew Ng specifically pointed out: among these capabilities, some are related to the LLM itself (such as function calling for tool invocation), but many are actually implemented by the framework the Agent runs on, external to the LLM. This is precisely the core value proposition of frameworks like LangChain and LangGraph.

Why Choose LangGraph?

Harrison Chase explained that the LangChain framework has long supported many elements including memory (in various forms), function-calling LLMs, and tool execution. As the agent paradigm evolved, LangChain specifically strengthened its support for Agentic Workflows, giving rise to LangGraph.

The course highlights several mainstream Agent paradigms, all of which receive better support through LangGraph:

• ReAct: An early agent-building paradigm combining Reasoning and Action. ReAct was proposed by Yao et al. in 2022 in the paper ReAct: Synergizing Reasoning and Acting in Language Models (published at ICLR 2023). Its core idea is to have the LLM generate a "Thought" trace before executing an action, then output a concrete Action, and finally observe the environment's feedback (Observation), forming a Thought→Action→Observation loop. This interleaving of reasoning and action significantly improves Agent performance on complex multi-step tasks and serves as an important foundation for understanding more complex Agent architectures. ReAct's key innovation lies in interweaving Chain-of-Thought (CoT) reasoning with external tool invocation: Observations come from real tool execution results (such as search engine returns or code execution outputs), enabling the LLM to correct its reasoning path based on real feedback rather than relying solely on parameterized knowledge—this is one of the key approaches to addressing LLM knowledge cutoff and hallucination problems;
• Self-Refine: A classic approach for implementing Iterative Refinement. Its core mechanism instantiates a single LLM as both a "Generator" and a "Critic," improving output quality through multiple feedback loops without requiring additional training data, leveraging the model's own metacognitive abilities to enhance generation quality;
• AlphaCodium: A cutting-edge programming Agent example built with "Flow Engineering," featuring a specialized pipeline designed for code generation tasks that includes problem understanding, test generation, and iterative repair, achieving significantly better results than direct prompting on competitive programming benchmarks.

A clear commonality is visible across the architecture diagrams of these paradigms: agents and their behaviors are fundamentally defined by a cyclical graph. This is precisely the origin of LangGraph's name—it uses graph structures to describe Agent state transitions, naturally fitting the Agent's "perceive—decide—act—feedback" loop.

LangGraph uses a Directed Graph data structure to orchestrate Agent workflows, with core concepts including Nodes, Edges, and State. Each node represents a processing unit (such as an LLM call or tool execution), edges define transition rules between nodes, and the globally shared State object persists throughout the graph's lifecycle, handling cross-node information transfer. LangGraph's design fundamentally draws from computer science concepts of Finite State Machines (FSM) and dataflow programming: each edge in the graph can carry conditional logic, forming Conditional Edges that enable Agents to dynamically choose their next action based on current state—this is the underlying mechanism for implementing cyclical patterns like ReAct and Self-Refine.

From a broader software architecture perspective, LangGraph's design philosophy has an interesting mapping to two microservice integration patterns: "Orchestration" and "Choreography." Traditional LangChain's chain structure is closer to the orchestration pattern—a centralized chain object explicitly controls the invocation sequence; LangGraph's graph structure is closer to state-machine-driven event choreography—nodes coordinate implicitly through state changes and conditional edges, making the expression of complex multi-branch, multi-loop workflows more natural. LangGraph's design also incorporates streaming processing and Checkpoint mechanisms, making it better suited for production deployment. Compared to LangChain's Chain structure, the graph structure natively supports conditional branching and loops, which is crucial for Agents requiring dynamic decision-making—compared to linear call chains, graph structures can more naturally express conditional branching, iterative loops, and multi-role collaboration.

LangGraph Course Learning Path

This course follows a progressive teaching approach that is quite beginner-friendly:

1. Building an Agent from Scratch: Hand-coding an agent using only an LLM and Python to understand the underlying principles;
2. Rebuilding as LangGraph Components: Re-implementing the same Agent with LangGraph to master each component's responsibilities;
3. Mastering Agentic Search: A dedicated module on Agentic Search capabilities and practical usage;
4. Two Advanced Capabilities:
   • Human Input: Also known as "Human-in-the-Loop" (HITL)—intervening and guiding the Agent at critical nodes. HITL is a design pattern for embedding human judgment into automated workflows. In Agent engineering, it typically manifests as pausing and requesting human confirmation before the Agent executes high-risk operations (such as sending emails, committing code, or calling paid APIs). From a system design perspective, HITL is essentially a trade-off mechanism between controllability and autonomy: a fully autonomous Agent is most efficient but carries the highest risk; a fully manual process is safest but loses the value of automation—HITL seeks the optimal balance by introducing human review at critical decision points. LangGraph natively supports HITL through its Interrupt mechanism, allowing developers to set breakpoints at any node in the graph, providing a safety valve for human review at critical decision points while preserving automation efficiency, effectively reducing risks from Agent "hallucinations" or operational errors. It's worth further noting that the granularity of HITL intervention is itself an engineering parameter requiring careful design—overly frequent human intervention degrades the Agent into an ordinary approval workflow, losing its autonomous value; too infrequent may result in loss of control at critical points. The industry is exploring the concept of "Adaptive HITL": the system dynamically adjusts the human intervention threshold based on the Agent's confidence scores, proactively requesting confirmation on low-confidence decisions while executing autonomously on high-confidence ones, achieving dynamic balance between efficiency and safety;
   • Persistence: Storing the Agent's current state for later recovery, which is extremely useful for both debugging and productionization. LangGraph implements persistence through a Checkpointer mechanism, supporting serialized storage of the Agent's complete state after each node execution to a database (such as SQLite, Redis, or PostgreSQL). This design delivers three major engineering values: fault recovery (restarting from the last checkpoint rather than from scratch), time-travel debugging (rewinding to any historical state node), and multi-session concurrency support. Notably, this checkpoint mechanism is highly similar to the Event Sourcing pattern in distributed systems—by recording every state change event rather than just storing current state, the system gains complete historical auditability and the ability to reconstruct state at any point in time. For complex Agent tasks that may run for minutes or even hours, persistence is a necessary condition for moving from prototype to production;
5. Comprehensive Hands-on Project: Completing a full end-to-end project with LangGraph.

Conclusion: The Current State and Future of Agent Development

From the dialogue between Andrew Ng and Harrison Chase, the maturation trajectory of agent technology is clearly visible: function calling has stabilized tool usage, agent-adapted specialized tools have resolved data format issues, and graph-structured frameworks like LangGraph provide engineering support for complex cyclical workflows.

Notably, the pace of Agent technology evolution is itself accelerating: as models like GPT-4o and Claude 3.5 continue to improve in tool calling accuracy and long-context processing capabilities, the "ceiling" for Agents is being continuously raised. Simultaneously, orchestration frameworks represented by LangGraph are standardizing capabilities that previously required extensive custom code (such as HITL, persistence, and multi-agent coordination) into reusable engineering components. The synchronous progress along these two dimensions is compressing the cycle for Agents to move from the lab to production from "years" to "months" or even "weeks."

From a longer-term perspective, the core challenge facing Agent engineering has shifted from "can it work" to "is it reliable"—specifically, how to ensure that Agent behavior is predictable, auditable, and intervenable in complex, open real-world environments. This has driven the rapid development of "Agent Observability" as an emerging engineering discipline: tools like LangSmith (LangChain's tracing platform) and Weights & Biases' Weave are beginning to provide full-chain tracing, performance analysis, and anomaly detection capabilities for Agent call chains, transplanting the three pillars of observability from traditional software engineering (logs, metrics, traces) into Agent systems. As Agents take on increasingly high-value tasks in production environments, the importance of observability infrastructure will rival that of orchestration frameworks themselves.

For engineers looking to get started with AI Agent development, this course covers core design patterns including planning, tool use, reflection, multi-agent collaboration, and memory, combines hands-on practice with LangGraph, and addresses two critical capabilities for engineering deployment: human-in-the-loop and persistence. It provides a complete path from theoretical understanding to engineering practice and is well worth the attention of developers aspiring to build intelligent agents.