Shared Selective Persistent Memory: Enabling Agentic LLM Systems to Reuse Context Across Sessions

The "Amnesia" Problem in Agentic LLMs

Large language model (LLM)-based agentic systems are becoming increasingly important tools in code generation. These systems handle complex tasks through multi-turn tool use, but they face a fundamental context challenge: every session starts from scratch.

Configuration choices, domain constraints, data schemas, and tool-use patterns accumulated in a previous session are completely discarded when a new session begins. This means the key information that made prior sessions efficient cannot be inherited — the agent is forced to repeatedly "relearn" the same background knowledge, resulting in significant efficiency losses.

Why You Can't Simply Save the Full History

An apparently straightforward solution would be to persist the complete conversation history and make it available to subsequent sessions. However, this approach is explicitly identified as both inefficient and counterproductive.

On one hand, saving the full conversation history is extremely wasteful in terms of tokens — lengthy history logs quickly consume the context window budget. On the other hand, a more serious concern is quality: irrelevant context degrades generation quality. When an agent is flooded with large amounts of unrelated information, it becomes harder to focus on the core requirements of the current task.

In short, more memory isn't always better — the key is to "remember what's worth remembering."

This touches on a fundamental engineering constraint of LLMs: the context window. Most mainstream LLMs have context windows measured in token counts (e.g., 32K, 128K, or larger), and all text fed to the model — including system prompts, conversation history, and tool outputs — consumes this finite budget. As history accumulates, it not only approaches the window limit but also drives up inference costs (which scale linearly with tokens). More critically, research has shown that LLMs suffer from a "lost in the middle" effect when handling very long contexts — models pay far more attention to information at the beginning and end of the context than to content in the middle. Mixing in large amounts of historical information dilutes the truly important instructions and constraints, causing generation quality to drop rather than improve. This explains why simply stacking history is not a viable solution.

The Core Idea Behind Shared Selective Persistent Memory

To address this tension, the authors propose a memory architecture called shared selective persistent memory, designed specifically for agentic systems.

The name itself reveals three design principles:

• Shared: Memory can be reused across different sessions, rather than being confined to a single interaction;
• Selective: The system doesn't blindly save everything — it deliberately identifies which contexts are worth retaining;
• Persistent: Selected contexts remain stable and available across sessions.

The core function of this architecture is to identify and retain four categories of reusable context, ensuring efficiency while preventing irrelevant information from degrading generation quality.

Four Categories of Reusable Context

The architecture focuses on four types of reusable context:

1. Task specifications: Descriptions of what the agent needs to accomplish — a structured distillation of task intent;
2. Data-related information: Such as data schemas, which help the agent understand the structure and constraints of the data it handles;
3. Domain constraints: Rules that must be followed within a specific business or technical domain;
4. Tool-use patterns: Tool invocation approaches that were validated as effective in prior sessions.

What these four categories share is cross-session stability and reuse value — they are not ephemeral fragments that are only temporarily useful within a single conversation. By contrast, one-off casual exchanges or transient intermediate reasoning steps generally don't qualify as information worth persisting.

Tool-use patterns deserve special mention, as they represent a memory type unique to multi-turn agentic systems. In Agentic LLM scenarios, agents typically have access to a set of callable external tools (such as code executors, database queries, API calls, etc.), and completing a complex task requires combining and invoking multiple tools in a specific order. These invocation sequences carry substantial implicit knowledge — for example, "you need to perform a format conversion before querying the database" or "a specific authentication parameter must be appended when calling a certain API." If these experiential call paths have to be rediscovered from scratch every time, it generates significant redundant trial-and-error overhead. Persisting validated tool-use patterns is essentially preserving the operational experience the agent has acquired through "practice" — analogous to the operational runbooks or best-practice documents that human engineers accumulate over time.

What This Design Means for Agentic Engineering

From an engineering perspective, shared selective persistent memory attempts to answer a question that is actively troubling agentic developers: How do you make an agentic system that gets to know you better over time, rather than requiring you to re-explain everything from scratch in every session?

This is especially true for code generation scenarios. Developers working on a project typically have a fixed tech stack, coding conventions, data structures, and common toolchains. If an agent can persistently remember these constraints, it can produce code that aligns with the project's conventions in subsequent sessions — reducing the repeated communication overhead.

The Balance That Selectivity Provides

The most noteworthy value of this approach lies in its pursuit of balance between "memory" and "conciseness." A completely memoryless system is inefficient, while a system suffering from memory overload degrades due to noise interference. Selective persistence attempts to find the middle ground — one that retains critical knowledge without polluting the context window — by classifying and filtering context before storing it.

This also reflects a consensus direction in agentic memory research: the key to memory architecture is not storage capacity, but the intelligence of retrieval and filtering.

In the field of agentic memory research, several mainstream memory management approaches offer useful points of comparison. RAG (Retrieval-Augmented Generation) dynamically retrieves relevant passages from an external knowledge base via vector search, addressing external knowledge acquisition — but it doesn't naturally handle cross-session context accumulation. Memory compression attempts to summarize and distill conversation history, retaining semantic essence while discarding details — but the compression process may lose structured constraint information. Shared selective persistent memory is closer in spirit to categorized memory storage — pre-defining which types of information are worth retaining, rather than processing all history indiscriminately. This classification-first design makes the filtering logic more interpretable and more amenable to engineering implementation and maintenance.

Summary and Observations

Shared selective persistent memory offers a structured approach to memory for multi-turn tool-calling agentic LLM systems. By explicitly categorizing reusable context into four types — task specifications, data information, domain constraints, and tool-use patterns — it avoids both the repetitive overhead of "starting from scratch" and the token waste and quality degradation caused by "saving everything."

It is worth noting that this article is based on the original abstract, and the specific implementation details, evaluation metrics, and experimental results have not been fully elaborated upon there. The performance of this architecture in real-world systems, as well as the mechanisms for memory updates and conflict resolution, remain to be validated by the complete paper. For researchers and engineers focused on agentic memory and long-term context management, this line of work represents a direction well worth tracking.