LangChain 1.2 Agent Memory Mechanisms: A Practical Guide to Short-term and Long-term Memory

Why Does Your Agent Keep 'Forgetting'?

When building AI Agents, many developers hit the same pitfall: the Agent can't remember previous conversations. You tell it "My name is John," and in the next turn when you ask "What's my name?", it draws a complete blank. This isn't because the model lacks capability — it's because the Agent's memory mechanism hasn't been properly configured.

This article is based on LangChain 1.2 and explains Agent short-term and long-term memory mechanisms from scratch. You'll understand why default Agents have no memory capability and how to give your Agent the ability to "remember conversations" through context management.

Basic Agent Creation in LangChain 1.2

In LangChain 1.2, creating an Agent has been significantly simplified compared to older versions. The core function is create_agent, which lets you quickly build an Agent with basic conversational capabilities.

Three Core Elements of Creating an Agent

An Agent's core capabilities can be summarized in three points:

1. LLM Reasoning Ability: The Agent relies on an LLM at its core for reasoning and decision-making
2. Tool Calling Ability: External tools available to the Agent are specified via the tools parameter
3. Memory Functionality: The Agent's ability to remember conversation history (the focus of this article)

The code structure for creating a basic Agent looks like this:

from langchain import create_agent

# Specify model and tools to create an Agent
agent = create_agent(
    model=deep_seek_model,  # The underlying LLM
    tools=[...]             # List of callable tools
)

The model is created using the init_chat_model method, which requires parameters like model name, provider, API Key, and Base URL. These configurations are typically managed centrally through environment variables.

The Problem with Memory-less Agents: An Intuitive Conversation Experiment

Basic Invocation Method

Once an Agent is created, you interact with it via the agent.invoke() method:

# First conversation
response = agent.invoke({"role": "user", "content": "我叫张三，你是谁？"})
print(response["messages"][-1].content)

The Agent's return structure contains a messages list, which stores the user's question (Human Message) and the AI's reply (AI Message) in order. To get the final reply content, simply access messages[-1].content.

A Clear Demonstration of Memory Loss

The critical problem appears during the second conversation:

# First conversation
response1 = agent.invoke("我叫张三，你是谁？")
print(response1["messages"][-1].content)
# Output: 你好张三！我是...

print("-" * 50)

# Second conversation
response2 = agent.invoke("我叫什么名字？")
print(response2["messages"][-1].content)
# Output: 我无法知道你的名字

Even though both calls are in the same piece of code and the program hasn't terminated, the Agent still can't recall the name you just mentioned. In the first conversation it clearly replied "Hello Zhang San," but in the second it says "I cannot know your name."

Why Does the Default Agent Have No Memory?

To understand this problem, you first need to understand the fundamental nature of large language models: LLMs are stateless inference functions. Each API call is a completely independent reasoning process — the model itself stores no session information. This is fundamentally different from traditional databases or stateful services. What we call "memory" is technically implemented by concatenating historical conversation text into the current request's Prompt, allowing the model to "see" the history within a single inference call, thus producing the effect of remembering.

Therefore, the default Agent has no memory mechanism configured. Each invoke call is a brand new, independent conversation from the Agent's perspective — context from the previous call is not automatically passed to the next one. It's like a person with amnesia who doesn't remember what happened before each time you meet. To solve this problem, you need to explicitly introduce a memory mechanism to the Agent.

Short-term Memory vs. Long-term Memory: Two Memory Mechanisms Explained

Agent memory mechanisms in LangChain fall into two major categories, suitable for different application scenarios.

Short-term Memory

Short-term memory, also called session memory or context memory, has the following key characteristics:

• Short lifecycle: Only valid during the current session; lost when the session ends
• Limited capacity: Typically constrained by the LLM's context window size (e.g., 4K, 8K, 128K tokens)
• Implementation: Historical conversation messages are passed as context with each request

The core problem short-term memory solves is: within a continuous conversation, the Agent can remember what was said earlier. This applies to scenarios like multi-turn Q&A and contextual understanding.

It's worth noting that the context window is the maximum number of tokens an LLM can process in a single call. Early GPT-3 only had 4K tokens, while modern models like GPT-4 Turbo support 128K tokens, and Gemini 1.5 Pro reaches up to 1 million tokens. As conversation turns increase, historical messages accumulate and gradually approach the window limit. In production engineering, strategies like message truncation (keeping only the most recent N turns) or summary compression (using an LLM to compress earlier conversations into summaries) are typically employed to manage context length and avoid errors or performance degradation from exceeding limits.

Long-term Memory

Long-term memory is persistent memory that spans across sessions, suitable for scenarios where the Agent needs to "know the user":

• Persistent storage: Memory survives even after program restarts or server reboots
• Larger capacity: Typically leverages external storage systems (databases, vector stores, file systems, etc.)
• Selective memory: Rather than remembering all conversation content, it extracts and stores key information

Long-term memory enables an Agent to remember user preferences and historical interaction records across multiple sessions. For example, if a user told the Agent last week that they prefer Python, the Agent will still know this preference when they return the following week.

In terms of technical implementation, long-term memory typically relies on vector databases (such as Chroma, Pinecone, Weaviate, Milvus, etc.) for semantic retrieval. The core principle is: conversation content is first converted into high-dimensional vectors by an embedding model (such as OpenAI's text-embedding-ada-002) and stored in the database; during retrieval, the current question is similarly converted to a vector, and the most relevant historical memory fragments are found via cosine similarity, then injected into the current request's Prompt. This approach is more intelligent than traditional keyword matching — it understands semantic-level similarity, meaning content can be retrieved even when different words are used, as long as the meaning is similar.

Practical Tips for Agent Development

Correctly Parsing the Return Structure

The Agent's return value is a dictionary containing a messages list, which stores in order:

1. The user's input as a Human Message
2. Possible intermediate tool call messages (Tool Message)
3. The final AI Message (reply content)

The standard way to get the final reply:

final_answer = response["messages"][-1].content

LangChain Version Compatibility Notes

Many enterprises are still using LangChain versions prior to 1.0, while this tutorial is based on version 1.2. The new version's create_agent API is more concise and intuitive, but the core concepts of memory mechanisms remain the same. If your project uses an older version, the Agent creation method will differ, but the implementation principles of short-term and long-term memory are consistent.

Summary: From Understanding 'Amnesia' to Building Memory Capability

This article demonstrated the default Agent's lack of memory through a simple conversation experiment. Key takeaways:

• In LangChain 1.2, use create_agent to quickly create an Agent
• The default Agent treats each call as an independent conversation without automatically preserving context
• To achieve continuous conversation capability, you must explicitly add a memory mechanism
• Memory is divided into short-term memory (in-session context management) and long-term memory (cross-session persistent storage)
• Short-term memory is limited by the LLM context window and requires truncation or summarization strategies to manage length
• Long-term memory relies on vector databases for semantic retrieval, preserving key user information across sessions

Understanding the fundamental question of "why the Agent has no memory" is essential before mastering how to add memory functionality to your Agent. This is an indispensable foundational step in building truly practical AI Agents.