Getting Started with LangChain: The Core Framework Connecting LLMs and AI Applications

Why Can't LLMs Build Applications Directly?

Amid the wave of AI Agent development, how to turn large language models into truly usable applications is a core challenge every developer faces. Based on the Agent development course series taught by a frontend deployment engineer on Bilibili, this article takes a deep dive into a key tool for Agent development—the LangChain framework—to help you understand its positioning and value within the broader AIGC industry chain.

What is an AI Agent? An AI Agent is an AI system capable of perceiving its environment, making autonomous decisions, and executing actions to achieve goals. Unlike traditional single-turn Q&A-style LLMs, Agents possess planning capabilities, memory mechanisms, and tool-calling abilities, enabling them to break down complex tasks into multiple steps and execute them sequentially.

Their underlying implementation typically relies on the ReAct (Reasoning + Acting) framework—proposed by Google's research team in 2022. Its core idea is to alternate between the model's reasoning traces (Thought) and concrete actions (Action), correcting subsequent decisions through observation (Observation) feedback. This "think → act → observe" loop allows the model to maintain logical coherence in complex tasks and avoid the short-sightedness of single-step reasoning.

From a deeper technical perspective, the ReAct framework resolves a fundamental contradiction: an LLM's knowledge is static (limited by its training data cutoff date), yet real-world tasks often require dynamic information (real-time search results, database queries, API responses). By decoupling "reasoning" from "acting," ReAct allows the model to pause during reasoning, call tools to fetch external information, and then incorporate the observed results into the next round of reasoning. This design later directly influenced OpenAI's Function Calling mechanism and Anthropic's Tool Use API—both of which are essentially engineering implementations of the proposition of "letting the model decide when and how to call external tools." For this very reason, building an Agent requires not just an LLM, but a complete engineering framework to coordinate collaboration across all stages.

Before formally getting to know LangChain, we need to clarify a fundamental question: since LLMs are so powerful, why can't we use them directly to build applications, and instead have to introduce an intermediate framework layer?

The course offers two core reasons.

First, there are a huge number of LLMs on the market, and each vendor's API interface differs from the others. Currently, mainstream LLMs include OpenAI's GPT series, Anthropic's Claude, Google's Gemini, Meta's LLaMA, as well as domestic Chinese models such as Ernie Bot, Tongyi Qianwen, and Zhipu AI. These APIs differ significantly in authentication methods, request formats, parameter naming, streaming output protocols, and more—for example, OpenAI uses Bearer Token authentication, while some domestic vendors use HMAC signatures; the definition of message role fields also varies across platforms.

This fragmented landscape is even more complex at the underlying technical level: context window lengths vary dramatically across models (from the early GPT-3.5's 4K tokens to the Claude 3 series' 200K tokens), the SSE protocol implementation details for streaming output differ, and the JSON Schema definition formats for Function Calling also contain vendor-specific extension fields. If developers want to support multiple models, they must adapt to each interface one by one—an enormous and hard-to-maintain workload. LangChain already has built-in support for a large number of mainstream models, abstracting away these underlying interface differences, allowing developers to call models from different vendors in a unified manner.

It's worth adding that this fragmentation problem is equally prominent in token billing standards: different vendors define "token" tokenization granularity differently (different tokenization algorithms like BPE and WordPiece mean that the same Chinese text may consume 20%-40% more or fewer tokens across different models). This introduces additional engineering complexity for cost estimation and context window management, and is an important part of the value proposition of a unified abstraction layer.

Second, even leading vendors like OpenAI—which have launched official Agent development tools and app stores (GPTs)—have not opened up their low-level APIs. Developers are largely confined to the idea level: they upload data, and the platform handles everything else.

This closed model brings obvious limitations: a lack of fine-grained low-level control makes precise customization difficult, and there are also numerous restrictions on file size, data volume, and so on. It may be suitable for quickly building demos, but it falls short for enterprise-grade AI development that requires deep customization.

The Core Challenge of Enterprise-Grade AI Development: The Principles of RAG

A key requirement in enterprise scenarios is securely handling private data. RAG (Retrieval-Augmented Generation) is the current mainstream solution, with vectorized retrieval at its core: enterprise documents are first converted into high-dimensional vectors via an embedding model (such as OpenAI's text-embedding-ada-002 or the open-source BGE series) and stored in a vector database (such as Pinecone, Weaviate, or Chroma). When a user asks a question, the question is likewise vectorized, and the most relevant document fragments are retrieved via cosine similarity or approximate nearest neighbor algorithms, then injected into the prompt context and handed to the LLM to answer.

The selection of a vector database is itself a complex engineering decision. It relies on approximate nearest neighbor (ANN) algorithms to enable fast retrieval of high-dimensional vectors—mainstream implementations include Qdrant and Weaviate based on HNSW (Hierarchical Navigable Small World) graph structures, as well as Faiss based on IVF (Inverted File Index) quantization compression. HNSW builds a multi-layer navigable small-world graph, quickly locating candidate regions from the sparse top layer at query time and then drilling down layer by layer to refine results, striking a good balance between recall and query latency; IVF, on the other hand, drastically reduces full-scan computation by clustering and partitioning the vector space. Different vector databases have different emphases regarding persistence capabilities, metadata filtering queries, multi-tenant isolation, and cloud-native support. Production environment selection requires a comprehensive evaluation of data scale (millions vs. billions of vectors), latency requirements (millisecond-level online retrieval vs. second-level offline analysis), and infrastructure costs.

Understanding the engineering complexity of RAG helps clarify why closed platforms struggle to meet enterprise needs. A complete production-grade RAG system involves far more than the two steps of "retrieval + generation": the document preprocessing stage requires handling PDF parsing (including multimodal content extraction from tables and images), Markdown/HTML structure preservation, encoding normalization, and more; chunking strategies significantly impact retrieval quality, with fixed-size chunking, recursive character chunking, and semantic chunking each having their applicable scenarios; the retrieval stage can introduce optimizations such as hybrid retrieval (dense vectors + sparse BM25 keyword retrieval), reranking models, and query rewriting; and the generation stage needs to handle citation tracking, hallucination detection, and answer confidence assessment. Every stage requires fine-grained parameter control—this is precisely the core difference that closed platforms cannot provide, while open frameworks like LangChain offer end-to-end tooling support.

Furthermore, when enterprises deploy AI applications, data security and compliance are unavoidable core constraints. On-premises deployment and public cloud API calls differ fundamentally in terms of data sovereignty, audit trails, and regulatory compliance (such as GDPR and China's MLPS 2.0). LangChain's native support for local inference frameworks like Ollama and vLLM enables enterprises to run LLMs in a fully isolated intranet environment, ensuring that sensitive data never leaves their borders or domains—a capability that closed platform solutions architecturally cannot provide, and the fundamental reason why many clients in finance, healthcare, and government scenarios choose the open-source framework route.

This workflow addresses two core pain points of LLMs: information lag caused by knowledge cutoff dates, and the inability to perceive private or real-time data.

LangChain's Positioning: The Glue Layer of the Tool Layer

Within the entire AIGC industry chain, LangChain is clearly positioned at the tool layer. As the course vividly puts it, its core role is to act as the "glue" between LLMs and applications (also called the glue layer).

Its value is primarily reflected in two dimensions:

Unified Encapsulation: Gluing Together Various LLMs

Faced with a fragmented model ecosystem, LangChain provides a unified abstraction layer. Whether the underlying call is to OpenAI, Anthropic, or various open-source models, developers can complete the call with similar code logic, greatly reducing the cost of multi-model compatibility and switching.

This abstraction follows the "Dependency Inversion Principle" (one of the SOLID design principles) in its engineering design: upper-layer business logic depends on the abstract interfaces defined by LangChain (such as BaseChatModel), rather than on specific vendors' SDK implementations. When you need to switch from GPT-4 to Claude 3, or introduce a lower-cost locally deployed model (such as Llama 3 running via Ollama) in production, you only need to modify the model initialization configuration—the business logic code requires no changes. This design is especially important in enterprise-grade development: vendor negotiations, model performance iterations, and cost control can all trigger the need for model switching, and a loosely coupled architecture significantly reduces migration costs.

From a cost-control perspective, the value of this abstraction layer is even more significant in production environments. Take a medium-scale enterprise knowledge base system as an example: the API call cost of GPT-4o may be more than 10 times that of GPT-4o-mini. Through LangChain's unified interface, teams can easily implement "routing strategies"—calling flagship models for complex reasoning tasks and downgrading to lightweight models for simple classification or information extraction tasks—compressing inference costs by 50%-80% without rewriting business logic. This model routing capability is difficult to implement elegantly by directly calling a single vendor's SDK.

Providing a Complete Agent Development Framework

LangChain's more important mission is to provide a complete framework and supporting tools for Agent development. The core philosophy of Agent development is—letting the LLM operate external systems, call tools, and interact with the real world. The LLM itself is just the "brain"; it needs "hands and feet" to actually get things done, and LangChain is precisely the framework that provides these "hands and feet" mechanisms.

LangChain's architecture consists of multiple functional modules: the Model I/O module is responsible for uniformly encapsulating various LLM call interfaces, internally handling conversational and completion models through the two abstractions ChatModel and LLM respectively; Prompt Templates provide structured prompt management, supporting variable interpolation, few-shot example injection, and message formatting, solving the engineering pain point of hard-coded prompts being hard to maintain; the Memory module provides short-term and long-term memory capabilities for conversations, with built-in strategies such as ConversationBufferMemory (complete conversation history), ConversationSummaryMemory (summarizing and compressing ultra-long conversations to alleviate token window limitations), and ConversationBufferWindowMemory (a sliding window retaining recent context); Chains allow multiple components to be linked into a processing pipeline; the Agents module implements the autonomous decision-making logic for tool selection and invocation; and Tools and Toolkits provide ready-made external capability interfaces for search, code execution, database queries, and more. Additionally, it includes built-in Document Loaders, Text Splitters, and Vector Stores components required for RAG, forming a complete enterprise knowledge base development toolchain.

It's worth mentioning that LangChain has also introduced two important architectural evolutions in recent years: LCEL (LangChain Expression Language) uses the pipe operator (|) to link components together, supporting streaming output, asynchronous calls, and batch processing, greatly enhancing the expressiveness of chain orchestration; LangGraph is a graph-structured orchestration tool launched for complex multi-Agent scenarios, allowing the definition of stateful looping workflows and making up for the shortcomings of linear Chains in scenarios requiring conditional branching and looping decisions.

The design of LCEL's pipe operator has profound engineering implications worth elaborating on. It draws on the combinator pattern from functional programming, making each LangChain component implement a unified Runnable interface—this interface stipulates that every component must implement four methods: invoke (single call), stream (streaming output), batch (batch concurrency), and ainvoke (asynchronous call). When components are chained together via the | operator, the runtime automatically selects the optimal execution path based on the invocation method: if the outer layer calls stream(), the entire chain automatically activates component-by-component streaming propagation, with no need for developers to manually handle generator protocols at each stage. This "write once, run in multiple modes" design greatly reduces the engineering cost of adapting the same business logic to different runtime environments such as synchronous APIs, WebSocket streaming, and background batch processing.

LangGraph's design draws on the mathematical model of directed graphs: each node represents a processing step (which can be an LLM call, tool execution, or conditional judgment), edges represent data flow, and state is passed between nodes and continuously accumulated. This makes complex workflows possible—such as Human-in-the-Loop review, Agent self-reflection and correction, and parallel subtask execution—which are precisely the scenarios that a linear Chain architecture cannot express elegantly.

Take a "self-reflection" workflow as an example: in LangGraph, you can define a loop structure where, after the Agent completes a step, a dedicated Critic Node evaluates the output quality. If it does not meet the preset criteria, a back edge is triggered to re-execute until the exit condition is satisfied (such as reaching a quality score threshold or the maximum number of iterations). This combination of loops and conditional branching is precisely the fundamental advantage of a directed graph structure over a linear chain—it directly maps the cognitive pattern of "the Agent needs to self-correct" into an executable computational graph topology.

The accompanying LangSmith platform focuses on observability, used to trace the input and output of each step in a chain call, specifically addressing the debugging difficulties of early versions.

Through LangChain, an LLM is no longer an isolated Q&A engine, but an actor capable of reading files, calling APIs, accessing databases, and executing code. This is precisely what distinguishes an AI Agent from a traditional conversational model.

LangChain Learning Path: Five Progressive Steps

The course breaks down LangChain learning into five parts, forming a clear knowledge structure.

Part One: What is LangChain and Its Development History. This traces the framework's evolution, helping learners build an overall understanding of the tool.

LangChain was created by Harrison Chase in October 2022, initially released as a Python open-source library—right on the eve of ChatGPT's launch, during the technology explosion period of LLMs. The framework quickly accumulated tens of thousands of stars on GitHub, becoming one of the de facto standards for AI application development. In 2023, the project completed a multi-million-dollar funding round, officially establishing the LangChain AI company, and successively launched commercial products such as LangServe (deploying Chains as REST APIs with one click) and LangSmith (an observability platform). The v0.2 version released in 2024 significantly restructured the package architecture, decoupling core abstractions (langchain-core), community integrations (langchain-community), and specific vendor packages (such as langchain-openai) into layers, to address the issues of bloated dependencies and frequent version conflicts in the early monolithic package.

This architectural restructuring had a clear engineering motivation: the early monolithic package design meant that installing LangChain would forcibly pull in dozens of third-party dependencies (including various model SDKs, vector database clients, etc.), even if a project only used a small portion of those features. In containerized production deployments, this directly led to bloated image sizes and dependency conflict risks. After the layered decoupling, a project using only OpenAI models needs only to install the two lightweight packages langchain-core and langchain-openai, greatly narrowing the dependency tree. Understanding this evolutionary history helps learners understand why a large number of online tutorial code snippets have API deprecation warnings, and how to correctly choose the best practices for the current version.

Part Two: What LangChain Can Do. This focuses on explaining the various capability modules of the framework, letting developers understand what tools it comprises and what responsibilities each module bears.

Part Three: Analysis of LangChain's Strengths and Weaknesses. This is a pragmatic and crucial section—the course clearly points out that LangChain is not a panacea. It has obvious advantages in some scenarios, but may not be applicable in others.

LangChain's Limitations and the Competitive Landscape

LangChain is not without controversy. As the framework's features continuously expanded, issues such as overly deep abstraction layers, debugging difficulties, and a steep learning curve have sparked wide discussion in the community. Some developers believe that LangChain's over-encapsulation actually reduces their sense of control over the underlying logic.

These criticisms have specific technical references: in early versions, a seemingly simple RAG query might pass through 5-6 layers of abstraction wrapping, making error messages difficult to locate as they are passed through the layers; the Callback mechanism, though powerful, presents comprehension barriers for developers unfamiliar with its event-driven, asynchronous nature; and frequent breaking changes also deterred some teams—this is precisely the fundamental reason why the LangSmith observability platform could gain market recognition as a standalone product: it solves the debugging difficulties created by the framework itself.

Examining this issue from a more quantitative angle: searching for "LangChain DeprecationWarning" on GitHub yields thousands of related issues, reflecting the systemic problem of insufficient API stability during the framework's rapid iteration phase. This is particularly unfriendly to enterprise-grade projects—while version pinning can mitigate risks in the short term, it cuts off access to security patches and performance optimizations, forming technical debt.

Meanwhile, several more targeted competing solutions have emerged in the ecosystem: LlamaIndex (formerly GPT Index) focuses on data indexing and RAG pipelines, with more fine-grained control over handling complex document structures, multi-hop retrieval, and knowledge graphs—its abstractions like QueryEngine and SubQuestionQueryEngine are specifically designed to optimize retrieval quality; AutoGen, led by Microsoft Research, introduces the "conversational programming" paradigm, where multiple Agents can collaborate to complete tasks through message passing—its role-division model of AssistantAgent and UserProxyAgent is particularly suited to automation scenarios of code generation and execution; CrewAI further simplifies the configuration approach for multi-Agent orchestration on top of AutoGen, lowering the entry barrier through declarative definitions of Role, Goal, and Task. Also worth noting is that OpenAI's Swarm framework, launched in 2024, and Anthropic's tool-calling API are pushing Agent development toward more lightweight, native SDK directions. In production environments, some teams also choose to directly call official SDKs and implement orchestration logic themselves, to obtain a more lightweight and controllable architecture.

Therefore, LangChain is more suitable as an entry-level framework for learning Agent development concepts, and understanding its boundaries is equally important.

Understanding this helps developers make rational judgments during technology selection, avoiding the blind application of frameworks.

Parts Four and Five: Hands-On Practice. Starting from environment setup through to completing the first example, these parts intuitively showcase each module and capability through code snippets, explained as development progresses.

Viewing the Boundaries of Tools Rationally

From the content organization of this chapter, it's clear that the course's approach is quite solid—it not only tells you what LangChain "can do," but also emphasizes "where its boundaries lie."

For engineers wanting to enter the field of AI Agent development, LangChain is an unavoidable and important tool. But as the course repeatedly emphasizes, the value of a tool lies in appropriate use. Understanding its essential positioning as "glue," appreciating its advantages over closed platforms in low-level control, multi-model compatibility, and enterprise-grade development, while soberly recognizing that it is not suitable for all scenarios—especially now that specialized frameworks like LlamaIndex and AutoGen can better meet specific needs—is the correct posture for mastering this technology.

A practical reference framework for technology selection: if a project's core requirement is complex document processing and fine-grained RAG pipelines, LlamaIndex is often the better choice; if you need to build an autonomous Agent system with multi-role collaboration, the role-division models of AutoGen or CrewAI are a better fit; if a team has extremely high requirements for low-level control and is willing to take on more infrastructure work, directly using the official SDK with a custom orchestration layer is the most lightweight path; and LangChain holds the most advantage in the middle ground of "needing to quickly integrate multiple capabilities while retaining flexible extensibility"—it is also currently the entry-level choice with the richest learning resources and the most mature community ecosystem.

From a learning-path perspective, another implicit value of LangChain lies in the fact that its modular design essentially constitutes a "conceptual map" of Agent development. Even if future projects shift to other frameworks, the Memory strategies, retrieval optimizations, tool-calling patterns, and Chain orchestration ideas learned in LangChain remain highly transferable—because these are essentially universal paradigms of AI application engineering, not implementations unique to LangChain. Using learning LangChain as an entry point to build a systematic understanding of the entire Agent development tech stack—this is the deeper logic behind the course's recommendation of this framework as a starting point.

Next, we will follow the course into the hands-on section, from environment setup to the first example, truly getting our hands on this core framework that connects LLMs and AI applications.

Key Takeaways

• The fundamental reason LLMs cannot build applications directly lies in: the fragmentation of vendor APIs (authentication methods, streaming protocols, and function-calling formats are all inconsistent), and the lack of low-level control capabilities on closed platforms
• LangChain's essential positioning is the "glue" of the tool layer, using a dependency-inversion abstraction design to shield underlying differences, and providing a complete toolchain for Agent development (Model I/O, Memory, Chains, Agents, Tools)
• RAG is the core paradigm of enterprise-grade AI development. A complete production-grade implementation covers multiple fine-grained stages such as document parsing, chunking strategies, hybrid retrieval, reranking, and hallucination detection. Among these, vector database selection (HNSW vs. IVF architecture) and data security compliance (private deployment capabilities) are key decision points for enterprise scenarios, for which LangChain provides end-to-end support
• LangChain is not a panacea. Issues such as overly deep abstraction layers, debugging difficulties, and frequent breaking changes have given rise to competing solutions like LlamaIndex (RAG specialization), AutoGen (multi-Agent collaboration), and CrewAI. Technology selection must be judged based on specific scenarios
• LangGraph and LCEL represent important evolutionary directions of the framework: the former supports complex workflows including loops and conditional branching (such as self-reflection and human-in-the-loop review) with a directed graph structure, while the latter enables the same business logic to automatically obtain three execution modes—synchronous, streaming, and asynchronous—through a unified Runnable interface and pipe operator. Together they make up for the limitations of the early linear Chain architecture
• Understanding tool boundaries is as important as mastering tool usage. LangChain's modular design also constitutes a "conceptual map" of AI application engineering, where core paradigms such as Memory strategies, retrieval optimization, and tool calling have cross-framework transferability—this is the core methodology running throughout the course