LangChain Hands-On Guide: From LLM Calls to Agent Development

Why We Need a Framework Like LangChain

Before diving into LangChain, let's first understand the inherent limitations of large language models (LLMs) themselves. LLMs are based on the Transformer architecture and acquire their language capabilities through self-supervised pretraining on massive amounts of text data—their core mechanism is next-token prediction: given a context, the model computes a probability distribution over every token in the vocabulary and samples to generate output.

The Transformer architecture was introduced by Google in the 2017 paper Attention Is All You Need, whose core innovation was the self-attention mechanism. It allows the model to attend to context at all positions simultaneously while processing a sequence, completely replacing the earlier RNN/LSTM paradigm that relied on sequential computation. Self-attention achieves global context modeling through the dot-product operations of three matrices—Query, Key, and Value: the output at each position is a weighted sum of the Value vectors across the entire sequence, with weights determined by the similarity between that position's Query and every position's Key. This all-to-all computation pattern gives the model powerful long-range dependency capture capabilities, but it also brings O(n²) computational and memory complexity—this is precisely the fundamental bottleneck in long-context processing. Flash Attention reduces the memory access complexity from O(n²) to O(n) through tiling and IO-aware optimization, and has become a standard optimization in mainstream inference frameworks today. GQA (Grouped Query Attention), on the other hand, has multiple Query heads share a single group of Key/Value heads, greatly compressing the memory footprint of the KV Cache while maintaining model quality, and is the mainstream architectural choice of new-generation models like Llama 3 and Mistral.

A token is the basic unit the model uses to process text, typically falling somewhere between a character and a word—for Chinese, one character is roughly equivalent to 1-2 tokens; for English, a word averages about 0.75 tokens. The vocabulary of GPT-series models usually contains around 50,000 tokens, and at each generation step the model computes a softmax probability distribution over the entire vocabulary, then jointly determines the diversity and determinism of the output through the temperature parameter and the Top-P sampling strategy. It's worth noting that the computational cost of this autoregressive generation process grows quadratically with sequence length (stemming from the all-to-all computation of the attention mechanism), which is the fundamental reason why inference speed and context length have become core technical challenges for current LLMs—optimization techniques such as Flash Attention and GQA (Grouped Query Attention) are all aimed at this problem.

This probabilistic nature means the model's knowledge is "frozen" at the cutoff point of its training data. For developers, the focus is not on studying the underlying algorithms, but on how to integrate the ready-made capabilities of LLMs into real-world business scenarios to build practical AI applications.

LLMs have three natural "weaknesses":

• Knowledge cutoff: The model only knows the internet knowledge up to its training cutoff; it cannot perceive events or new knowledge that occurred afterward.
• No memory: Standard Transformers have no persistent state during inference—every call is a stateless forward computation. If you tell it "My name is Zhang San," it won't remember in the next turn of conversation. The "memory" we usually experience is actually maintained by external systems managing the context.
• No access to business data: By default, the model cannot read private business data; it needs to invoke external knowledge by binding tools. The mainstream technical approach for solving this problem is Retrieval-Augmented Generation (RAG): private documents are chunked and converted into high-dimensional vectors through an embedding model, then stored in a vector database (such as Chroma, Pinecone, or Milvus); when a user asks a question, semantically similar document fragments are retrieved first and injected into the prompt as context, letting the model answer based on real documents. This approach avoids expensive fine-tuning while enabling real-time updates to the knowledge base, making it the preferred solution for scenarios like enterprise knowledge bases and customer service bots.

The working principle of vector retrieval is worth understanding in depth: embedding models (such as OpenAI's text-embedding series or the domestic BGE series) map text into a high-dimensional semantic space (typically 768 to 4096 dimensions), where semantically similar text has a smaller Euclidean or cosine distance in that space. When storing, vector databases pre-build an Approximate Nearest Neighbor Index (ANN Index), with common algorithms including HNSW (Hierarchical Navigable Small World, a graph-based hierarchical navigation structure) and IVF (Inverted File Index). HNSW builds a multi-layer graph structure enabling hierarchical navigation from coarse to fine granularity—queries start from the sparse top-layer graph to quickly locate candidate regions, then search precisely in the dense bottom-layer graph, balancing recall and query speed, making it the most widely used ANN algorithm in production environments today. During retrieval, there's no need for brute-force enumeration over the entire database; instead, graph traversal or inverted lists quickly locate the Top-K similar vectors, and the retrieval latency for a typical million-scale document set can be kept at the millisecond level. This mechanism allows RAG systems to maintain low query latency even as the knowledge base scales up, and is the key engineering guarantee that makes it viable for enterprise production environments.

The document chunking strategy is equally important—overly long chunks introduce noise, while overly short ones lose context. Common strategies include fixed-window chunking, recursive character chunking (RecursiveCharacterTextSplitter), and semantic-boundary-based chunking. Recursive character chunking splits by paragraph, sentence, and character priority in a layered manner, offering the most balanced preservation of semantic integrity, and is LangChain's officially recommended default strategy; semantic-boundary-based chunking uses an embedding model to detect points of abrupt semantic distance change between adjacent sentences, more accurately preserving logically complete knowledge units, but at higher computational cost. Different chunking strategies directly affect the accuracy of the final answer and are an important step in RAG engineering tuning.

A common misconception among developers is treating the model as an "omniscient" black box, when in reality it's more like a well-read but isolated librarian: rich in knowledge but unable to access information outside the library.

It is precisely to systematically solve these problems that AI application development frameworks came into being. LangChain (along with its companion LangGraph) helps developers uniformly manage memory, tool calls, context, and state, so these capabilities can be used directly without being implemented from scratch.

LangChain's Positioning and Core Features

LangChain was founded by Harrison Chase in October 2022 and is one of the earliest open-source frameworks to systematize LLM application development. Its core contribution lies in abstracting the originally fragmented work of LLM integration into a unified interface, and providing high-level abstractions such as Chain, Agent, and Memory. In the competitive landscape, similar frameworks include Microsoft's Semantic Kernel (focused on the .NET/C# ecosystem), LlamaIndex (focused on RAG and data indexing), and emerging multi-agent frameworks like CrewAI and AutoGen. LangChain's biggest advantage is its community ecosystem and the number of integrations—as of 2024 it has integrated more than 70 model providers and hundreds of external tools.

LangGraph is a stateful multi-agent orchestration framework launched by the LangChain team. It describes the execution flow of agents through a directed graph (DAG), addressing the shortcomings of the original LangChain in controlling complex workflows. Compared to the problem of early Agents easily falling into infinite loops, LangGraph precisely controls execution flow through nodes and explicitly defined edges, allowing developers to clearly set loop termination conditions and making the orchestration of complex multi-step tasks more reliable.

LangGraph's graph scheduling mechanism is worth elaborating in detail: its core data structure is a Stateful Directed Graph. Each node is an executable Python function that receives the current graph state as input and returns a state update; edges are divided into normal edges (unconditional flow) and conditional edges (which dynamically decide the next node based on state), with the latter being key to implementing branch decisions and loop control. The graph's global state is defined via TypedDict, and all nodes share and modify the same state object, which means information passing across nodes doesn't require explicit parameter passing and also makes it easy to save and restore execution state at any checkpoint—this is especially important for scenarios requiring human review (Human-in-the-loop). The checkpoint mechanism supports multiple persistence backends at the underlying level (such as SQLite, PostgreSQL, and Redis), enabling long-running Agent workflows to resume from any checkpoint after interruption, which has significant engineering value for complex task orchestration that may take hours or even days to complete. This design draws on the Reducer concept from functional programming, where each node's output is a "patch" to the state rather than a full replacement, thereby supporting state merging across parallel nodes. Compared to the linear chaining of the original LangChain, the graph structure can naturally express complex control flows like loops and parallel branches, making it important infrastructure for building production-grade Agent workflows.

LangChain is essentially a framework for developing AI applications, in the same category as the OpenAI SDK and Claude SDK. But in the actual market, LangChain has the highest adoption rate and the most mature ecosystem.

Why Choose Python

Python dominates AI development (adopted by over 90% of companies), for two main reasons:

1. A necessity for on-premise deployment: Local deployment of LLMs almost always relies on the Python ecosystem. Take vLLM as an example—a high-performance LLM inference engine developed by UC Berkeley, whose core innovation PagedAttention borrows the virtual memory paging concept from operating systems to manage the KV Cache. Traditional LLM inference frameworks face severe memory fragmentation when handling concurrent requests—the KV Cache for each request (the key-value cache used to store intermediate results of the attention mechanism) requires pre-allocated contiguous memory, causing large amounts of memory to be wasted because it can't be reused, and GPU utilization is usually below 40%. PagedAttention splits the KV Cache into fixed-size physical blocks and maps logical to physical addresses via a block table, nearly eliminating memory fragmentation and boosting GPU utilization to over 90%. Combined with the Continuous Batching strategy—dynamically inserting new requests into the batch being executed without waiting for the whole batch to finish—vLLM can achieve up to 24x higher throughput than the native Hugging Face implementation on the same hardware, making it the most mainstream LLM inference engine in the open-source community today. A typical on-premise deployment stack is: vLLM inference engine + FastAPI wrapping an OpenAI-compatible interface + Python business logic layer. Although the Java ecosystem has community projects like Langchain4j, it lacks mature toolchains for underlying capabilities such as model quantization, GPU scheduling, and tensor parallelism.
2. Ecosystem first: LangChain/LangGraph is one of the earliest AI application frameworks, and the API launches of almost all new models prioritize adapting to its ecosystem. Furthermore, vector database clients, embedding model libraries (such as sentence-transformers and the BGE series), and data processing tools (such as LangChain's DocumentLoader system) all treat Python as a first-class citizen, resulting in significant ecosystem synergy.

Two Core Features

A unified model interface is where LangChain's most practical value lies. In the past, calling LLMs from different vendors required dealing with separate interfaces in varying formats, and switching models meant rewriting the code each time. With a unified framework, developers only need to swap the model name, leaving the underlying compatibility work to the framework.

A modular architecture breaks complex applications into independent modules—state, context, message history, tool calls, prompts, middleware, and so on, each handling its own responsibilities. Agents, memory management, and RAG retrieval all fall under this modular system, making the construction of complex applications clear and controllable.

Environment Setup and Model Configuration

For the development environment, this course uses PyCharm (Python 3.13) as the IDE, with LangChain version 1.3.2 and LangGraph version 1.2.x. Installing PyCharm simply requires downloading the latest version and double-clicking to install, then configuring the Python environment.

Configuring API Keys

The project manages keys through a .env file, and only two items need to be configured at the core:

DEEPSEEK_API_KEY=your_key
DEEPSEEK_BASE_URL=interface_address

The key is created on the API Keys page of the DeepSeek website, and the BaseURL can be found in the interface documentation. The code loads the .env file through the dotenv library, reading the configuration as environment variables for subsequent use. Of course, DeepSeek is just an example—developers can also choose models from vendors like OpenAI, Anthropic, Tencent Hunyuan, Alibaba Qwen, or Zhipu.

Calling the LLM: init_chat_model

LangChain provides a universal model initialization method, init_chat_model, which is the most recommended way to connect to an LLM:

from langchain.chat_models import init_chat_model

deepseek_llm = init_chat_model(
    model="deepseek-v4-pro",
    model_provider="deepseek",
    api_key=api_key,
    base_url=base_url,
    extra_body={"thinking": {"type": "disable"}}
)

Weighing the Trade-offs of Thinking Mode

The setting of the thinking parameter deserves special attention. DeepSeek's "thinking mode" corresponds to the explicit activation of Chain-of-Thought (CoT) reasoning capabilities.

CoT was proposed by Google Research in 2022, with the core finding that adding a guiding phrase like "let's think step by step" to the prompt can significantly improve the model's accuracy on complex tasks such as mathematical reasoning and logical inference. Reasoning models (such as DeepSeek-R1 and OpenAI's o1/o3 series) internalize this idea into the training stage—using reinforcement learning to let the model learn to autonomously unfold an internal reasoning process before answering. This internal thinking is wrapped in special tags (such as <think>...</think>) and is not directly presented to the user, but does consume the token quota.

It's worth noting that DeepSeek-R1 used the GRPO (Group Relative Policy Optimization) algorithm instead of traditional PPO during training. The core difference between the two lies in how the reward baseline is computed: PPO relies on a separately trained value network (Critic Network) to estimate the expected value of each state, while GRPO samples multiple answers to the same question and computes their relative in-group scores as the baseline, eliminating the training overhead of the value network and greatly reducing the parameter count and memory footprint. This innovation achieved results close to OpenAI o1 on benchmarks such as math competitions and code generation, but reportedly at only about 1/20 of the training cost, allowing the open-source community to reproduce R1's training process with relatively manageable computational resources—which is the core reason for the widespread attention it received in the community.

Reasoning models output an internal "thinking process" before generating the final answer, and this mechanism significantly improves accuracy on complex reasoning tasks such as math and logical inference, at the cost of greatly increased latency (thinking tokens consume additional generation time and billed token counts).

DeepSeek's new models are divided into thinking mode (corresponding to the old Reasoner, such as the Flash version) and non-thinking mode (corresponding to the old Chat, such as the Pro version). Here it's recommended to disable thinking mode, for two reasons:

• Building agents and regular LLM applications usually doesn't require deep reasoning, and enabling it noticeably slows down the response speed;
• The LangChain framework's current compatibility with thinking mode is not yet mature—the framework parses model output in a standard format by default, and thinking tokens can interfere with the normal parsing of the message structure, potentially causing exceptions.

For most scenarios, the Flash version is already sufficient; even when using the Pro version with stronger reasoning capabilities, there is no need to enable thinking mode.

Initiating a Conversation

Calling the LLM uses the invoke method:

response = deepseek_llm.invoke("Please introduce yourself")
print(response)
print(type(response))

The returned result is Markdown-formatted text. Printing it directly gives mediocre readability, but after Markdown rendering the display effect is excellent—this is also key to how the pages of subsequent projects are presented.

Understanding Message Types

The object type returned by the call is AIMessage, which is the entry point to understanding LangChain's message system. LangChain's message type system originated from the role design of the OpenAI Chat Completions API (system/user/assistant) and extended it. Under the langchain_core.messages module, there are four main types of messages:

• SystemMessage: The system prompt, used to set the model's role and behavioral boundaries. This is a key means in prompt engineering for setting the model's "persona" and behavioral constraints, usually containing instructions like role definitions, output format requirements, and safety boundaries. The content of the system prompt has a huge impact on model behavior; an excellent system prompt design can make the same base model exhibit completely different styles and capability boundaries. In multi-turn conversations, the SystemMessage always sits at the top of the message list, and its weight in the model's attention mechanism is usually higher than that of ordinary user messages—some research refers to this phenomenon as "system prompt priority," which is also why jailbreak attacks often need to bypass system prompt constraints. From the perspective of the attention mechanism, the system prompt is located at the head of the sequence and has direct attention connections with all subsequent tokens in the self-attention computation, which architecturally guarantees its continuous influence over global generation behavior.

• HumanMessage: The message input by the user. When you pass in "Please introduce yourself," the framework automatically wraps it into a HumanMessage under the hood.

• AIMessage: The reply message returned by the model. When the model decides to call a tool, the AIMessage also carries a tool_calls field containing structured information about the tool name and parameters—this is the core embodiment of the LLM's Function Calling capability. OpenAI officially introduced Function Calling in June 2023; its essence is injecting a large number of "tool call examples" during the model's supervised fine-tuning (SFT) stage, teaching the model to recognize which user intents require calling external tools and to output structured parameters conforming to the JSON Schema specification. In 2024, Parallel Function Calling was further introduced, allowing the model to declare multiple tool calls in a single response—the model automatically identifies dependencies between tasks and issues call instructions in parallel for subtasks that are independent of each other, reducing the execution latency of multi-step serial Agents by more than 50%. Major model vendors (including Anthropic Claude, Google Gemini, Baidu ERNIE, and Alibaba Qwen) subsequently followed suit, forming a de facto industry standard centered on describing tool parameters with JSON Schema. LangChain encapsulates the differences among vendors through a unified bind_tools interface, automatically converting the type annotations and docstrings of Python functions into the JSON Schema format required by the respective vendor at the underlying level, allowing the same set of Agent code to seamlessly switch between underlying models.

• ToolMessage: The message produced by a tool call, one of the core concepts when building Agents. The result after a tool executes is returned to the model in the form of a ToolMessage, forming a complete tool-call loop. This loop is the implementation basis of the ReAct (Reasoning + Acting) framework—the model alternates between reasoning (Thought) and action (Action) until the task is completed. Specifically, a complete tool call involves three stages: ① the model outputs an AIMessage with tool_calls, declaring the tool and parameters to be called; ② the framework executes the corresponding tool externally (such as a search API, database query, or code executor); ③ the execution result is wrapped as a ToolMessage and returned to the model, which synthesizes all context to generate the final reply. This mechanism transforms the model from "only able to talk" to "able to act," serving as the key bridge between the LLM and the real world.

This set of message types is essentially a structured representation of multi-turn conversation history—an Agent's "memory" is these message lists serialized and stored, then injected into the model as context at each call. Understanding the relationships between them is an important cognitive leap from simple LLM calls to Agent development.

A Note on the Model-Class Calling Approach

Besides the universal init_chat_model, the new version of LangChain also supports creating instances via model classes, such as ChatDeepSeek, ChatOpenAI, and other dedicated classes. However, it's recommended to prioritize the universal approach, because it is naturally compatible with models from various vendors and has a low switching cost. Only when using custom models or special models not supported by the universal approach do you need to consider dedicated model classes.

Summary: The First Step from LLM to Agent

This article lays the foundation for hands-on work with LangChain: we understood the three major limitations of LLMs (knowledge cutoff, no memory, and no access to business data), clarified LangChain's positioning and value as an AI application framework, mastered the configuration and calling of the unified model interface, and became familiar with the core message type system.

These seemingly basic contents are actually the foundation for building Agents and RAG projects. What the LLM solves is "being able to talk," while what the Agent needs to solve is "being able to do things"—the difference between the two is: the LLM is merely a static probabilistic prediction engine, while the Agent, through tool calls, memory management, and multi-step reasoning, gives the model the ability to interact with the external world and complete complex tasks. Having mastered model calls and the message system, you can now move on to the advanced development stages of tool calling, memory management, and agent orchestration.

Key Takeaways

Key Takeaways