MemGAS: Multi-Granularity Memory Association for Precise AI Agent Recall

When a user has been chatting with an AI Agent for months, what exactly should the system remember? Entire conversations, every single detail, summaries, or keywords? The MemGAS paper from ICLR 2026 provides a clear answer: you can't just pick one. This paper proposes a multi-granularity memory association and adaptive selection framework that enables conversational Agents not only to "remember" but, more critically, to "recall at the right granularity level."

The Dilemma of Single-Granularity Memory

Existing conversational memory systems typically follow the RAG approach, chunking conversation history into sessions, turns, summaries, or topic chunks, then performing vector retrieval. Some methods further build tree or graph structures, but they usually still operate around a single level—such as using only entity graphs or only summary indices.

The problem is that user queries frequently span across sessions, topics, and time. A single granularity can hardly preserve both details and relationships simultaneously. For example, when a user asks "How many online courses have I completed in total?" if the system only stores session-level summaries, it might only find coarse-grained information like "learning plans and career advice" while losing specific course counts. Conversely, if only turn-level details are stored, the system might only hit one conversation and miss information scattered across different sessions.

This paper comes from the University of Science and Technology of China, City University of Hong Kong, Huawei, and other institutions. Core authors include Xu Derong and others. The paper is titled "From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents."

MemGAS's Four Memory Granularity Levels

The core design of MemGAS is to simultaneously maintain four granularity levels of memory units:

• Session-level: Retains an entire conversation, suitable for viewing complete background and context
• Turn-level: Retains specific turns, suitable for finding precise information like dates, names, and quantities
• Summary-level: Uses LLM to compress topics, providing medium-granularity semantic summarization
• Keyword-level: Uses keywords and entities as precise anchors for quick localization

But the key isn't just "storing a few more indices." MemGAS's real innovation lies in its memory association mechanism: when new conversations come in, the system encodes different granularities into vectors, compares similarity with historical memories, and then uses a Gaussian Mixture Model (GMM) to split historical memories into "relevant sets" and "irrelevant sets." Only relevant old memories establish edges with new memories, entering the association graph.

Why GMM Instead of a Fixed Threshold?

GMM solves the problem of "which old memories are worth connecting." If you simply cut by a fixed threshold, results are easily affected by data distribution. GMM treats similarity scores as two types of distributions—one resembling relevant memories, the other resembling irrelevant memories—so new memories won't blindly connect to all historical nodes but only strengthen the more credible associations.

Entropy Router: Adaptive Selection of Optimal Memory Granularity

When a query arrives, how does MemGAS decide which granularity to use? It employs an Entropy Router.

The system separately computes the similarity distribution of the query across session, turn, summary, and keyword granularities. If a certain granularity's distribution is concentrated (low entropy), it indicates a more certain match and receives higher weight; if the distribution is scattered (high entropy), it indicates uncertainty and receives lower weight.

An intuitive example: when a user asks "How many online courses have I completed in total?" the turn-level might precisely hit "three Coursera courses" and "two edX courses," while the session-level would bring up lots of learning plans and career advice—in this case, fine granularity is more certain and gets higher weight. Conversely, if the user asks "What have I mainly been planning recently?" session or summary might be more appropriate, as they can provide complete contextual narrative.

Graph Propagation: From Precise Retrieval to Associative Recall

Ordinary TopK retrieval only looks at what's most similar to the query, while MemGAS introduces Personalized PageRank (PPR) for propagation on the association graph.

The specific process: first use weighted similarity to select several seed nodes, then perform PPR diffusion on the graph. This way, once a keyword is hit, sessions, turns, and summaries stably connected to it all have a chance of being surfaced—more like associative recall rather than simple pattern matching.

A typical case from the paper: a user mentioned completing three Coursera courses in one session and two edX courses in another. When asked about the total, retrieving only the first session would answer "three," but MemGAS's multi-granularity association can recover both sessions through shared topics and keywords, yielding the correct answer of "five."

Balancing Expansion and Narrowing in Graph Propagation

Could graph propagation also bring in irrelevant content? This risk indeed exists. So MemGAS also has a final LLM filtering layer that presents TopK candidate memories along with the query to an LLM, which removes duplicates, irrelevant content, or noise, passing only the most critical context to the answering model. In other words: the graph is responsible for expanding recall, filtering is responsible for narrowing context.

Two key hyperparameters validated in the paper: the number of seed nodes works best around 15, and the entropy temperature parameter is most stable around 0.2. The point isn't fixed values, but ensuring that association can spread without spreading boundlessly.

Experimental Results: MemGAS Comprehensively Outperforms Mainstream Baselines

The paper evaluates on four long-term memory benchmarks: Locomo, LongMemEval-S, LongMemEval-M, and LongMTBench Plus, with baselines including FullHistory, Contriever, SEACOM, HIPPO RAG, RAPTOR, AMEM, and other mainstream methods.

Several representative numbers:

Dataset	Metric	MemGAS	Strongest Baseline
LongMemEval-S	F1	20.38	14.73 (HIPPO RAG)
LongMemEval-S	Recall@3	78.51	75.53 (HIPPO RAG)
LongMemEval-M	F1	16.85	11.88 (Contriever)
LongMTBench Plus	F1	41.49	36.07 (FullHistory)
Locomo	Recall@10	81.82	Next best baseline

Ablation experiments are also straightforward: complete MemGAS achieves an F1 of 20.38, dropping to 13.78 after removing all core components; individually removing GMM, PPR, memory association, or the entropy router all cause performance degradation, demonstrating that each module makes an independent contribution.

Computational Cost and Resource Overhead Analysis

The paper concludes that costs are manageable. On LongMemEval-S, building memory processes approximately 52.9M input tokens, close to the raw corpus scale; the additional memory from Summary and Keyword is about 27MB, roughly 10% of the original memory; and the retrieval latency introduced by core modules is also minimal.

Limitations and Real-World Deployment Challenges

Despite MemGAS's excellent experimental performance, it still faces several key challenges in real products:

1. Stability of summary and keyword extraction: Emotional expressions, implicit intents, or non-factual fragments may cause unstable LLM extraction
2. False association risk: High similarity doesn't equal semantic relevance—the graph structure may produce erroneous connections
3. Long-term operational costs: As memory continuously grows, privacy management, permission control, forgetting mechanisms, conflict updates, and real-time cost control are all issues that must be addressed

Conclusion: A Paradigm Shift from "Remembering" to "Recalling"

MemGAS advances the structural paradigm of long-term memory systems: don't treat conversation history as a pile of text chunks waiting to be retrieved, but decompose it into different granularities, establish credible associations, and then adaptively select and propagate based on the query.

It differs fundamentally from knowledge graph RAG and skill graphs—knowledge graphs center on entity relationships, skill graphs center on execution dependencies, while MemGAS's memory association graph has nodes backed by conversational memories at different granularities. It's not responsible for planning actions but for finding evidence with both detail and context in long-term conversation history.

For conversational Agents, "remembering" isn't enough—the key is recalling at the right granularity level.