CLAM: Single Forward Pass Enables Long-Term Memory Weight Consolidation for LLMs

CLAM compiles conversational activation streams into low-rank adapters via a hypernetwork, achieving gradient-free LLM long-term memory consolidation.
CLAM introduces a dual-channel LLM long-term memory solution combining parameterized consolidation with an episodic buffer. Its recurrent hypernetwork compiles session activation streams into a low-rank adapter in a single forward pass, updating model weights instantly without gradient steps or generation calls; the episodic buffer supplies verbatim details at inference. A gated latent state carries knowledge across sessions at fixed size, preventing model inflation. On LoCoMo and LongMemEval-S, CLAM leads agent baselines by 1-45 points and gradient methods by 10-45 points, cutting memory construction time by 3-151x. Ablations confirm parameterized consolidation complements episodic memory, offering an efficient new paradigm for agent long-term memory.
Long-term memory in large language models (LLMs) has always been a key bottleneck for agent systems. Traditional agent memory solutions extract conversational experience into text and retrieve it at query time, but the model itself does not evolve—no matter how many interactions occur, every query starts from the same set of weights. This decoupled design leads to accumulated extraction errors, implicit cross-session patterns that cannot surface, and construction costs that rise linearly with conversation length. On the other hand, fine-tuning on historical interactions can change the weights, but it requires backpropagation at serving time and degrades in performance with long histories. The recently proposed CLAM (Carried Latent Adapter Memory) attempts to combine the strengths of both, offering an instantaneous weight-space consolidation solution for LLM long-term memory that requires no gradient steps and no generation calls.

The Predicament of Existing Memory Solutions
Current mainstream agent memory systems fall into two categories: retrieval-based approaches based on text extraction and gradient-based approaches based on parameter fine-tuning. Retrieval-based solutions compress interaction experience into text snippets and provide context at inference time through retrieval-augmented generation (RAG). This approach is simple to implement but has obvious flaws: every query starts from the same model weights, so the model itself does not learn from history; the extraction process may introduce errors, and these errors compound and amplify in subsequent interactions; more importantly, certain implicit cross-session patterns cannot be fully expressed by any single text snippet, and retrieval mechanisms struggle to capture this global structure.
Gradient-based solutions attempt to update model weights by fine-tuning on historical interactions. In theory, this allows the model to truly "remember" experience, but in practice it faces the high computational cost of backpropagation at serving time, and as history length increases, fine-tuning effectiveness degrades—the model may forget old knowledge or overfit to recent data. Both approaches have limitations, and there is an urgent need for a new method that achieves parameterized consolidation without online gradients.
Retrieval-Augmented Generation (RAG) is a technical paradigm that combines information retrieval with text generation: the system first retrieves document snippets relevant to the current query from an external knowledge base, then injects these snippets as context into the LLM's prompt to guide the model toward more accurate answers. In agent memory scenarios, RAG is often used to recall relevant experience from historical conversations. However, the essence of RAG is "conditioned generation" rather than "parameterized learning"—model weights remain unchanged, and every inference still starts from the same initial state, so it cannot internalize implicit cross-session patterns, and retrieval quality depends heavily on the accuracy of text extraction.
CLAM's Core Design: Dual-Channel Memory
CLAM's innovation lies in combining parameterized consolidation with an episodic buffer into a dual-channel memory system. At its core is a recurrent hypernetwork that performs parameterized consolidation: it compiles a stream of session activations into a low-rank adapter, and the entire process requires only a single forward pass—no gradient steps and no language model generation calls. This means the model can update its weights instantaneously without interrupting service, and the update cost is independent of history length.
On the other side, a complementary episodic buffer provides verbatim details at inference time. Parameterized memory and episodic memory each serve their own function: the former captures abstract cross-session patterns, while the latter preserves specific facts. A gated latent state carries knowledge across sessions at a fixed size, allowing parameterized memory to accumulate continuously without growing—this solves the problem in traditional fine-tuning where model size inflates with history.
Hypernetwork is a class of networks specifically designed to generate the weights of another neural network. Unlike traditional networks that directly output predictions, a hypernetwork receives some input (such as a task description, activation stream, or context vector) and outputs some or all of the parameters of a target network. The recurrent hypernetwork in CLAM receives a stream of session activations and outputs the weights of a low-rank adapter, thereby achieving "instant weight updates" without performing backpropagation. This design transforms "learning" from gradient optimization into a single forward generation, greatly reducing the computational overhead of online consolidation.
Low-Rank Adapter originates from the field of parameter-efficient fine-tuning (PEFT), whose core idea is not to directly modify all weights of a pretrained model but to insert a small number of trainable parameters. Specifically, for a given weight matrix W, the adapter represents its update as the product of two low-rank matrices ΔW = A·B, where the ranks of A and B are much smaller than the dimensions of the original matrix. This greatly reduces the number of trainable parameters while maintaining strong expressiveness. CLAM uses a hypernetwork to generate such low-rank adapters, which both controls the storage overhead of memory and avoids destructive modifications to the original model weights.
Experimental Performance: Leading Across the Board
On two long-term memory benchmarks, LoCoMo and LongMemEval-S, CLAM was evaluated on three open-source models (2-7B parameters). Results show that CLAM leads all agent baseline methods by 1-45 percentage points and all gradient-based parameterized methods by 10-45 percentage points. More critically, CLAM requires no generative LLM calls and no gradient steps when building memory, and its actual runtime is 3-151 times less than extraction-based systems. This efficiency advantage makes CLAM highly feasible in real serving scenarios.
The complementarity of the dual channels was validated in ablation experiments: when the episodic buffer fails to hit the golden evidence, the consolidated adapter still delivers a 12-15 percentage point accuracy improvement; even with an oracle buffer, the adapter still contributes a 4-5 percentage point gain. This shows that parameterized consolidation is not a substitute for episodic memory but a powerful complement.
Implications for LLM Memory Architecture
The significance of CLAM lies not only in performance numbers but also in the new memory consolidation paradigm it proposes: compiling activation streams into weight updates on the fly via a hypernetwork, bypassing the online gradient bottleneck of traditional fine-tuning. This idea of "consolidation in a single forward pass" provides a viable path for LLMs to achieve true parameterized learning during continuous interaction. For agent applications that require long-term memory (such as personal assistants, customer service chatbots, and continual learning systems), CLAM demonstrates a solution that balances efficiency and effectiveness.
Of course, CLAM has currently only been validated on open-source models with 2-7B parameters, and its performance on larger-scale models remains to be studied. In addition, issues such as the capacity limits of low-rank adapters and the information bottleneck of the gated state may also surface in more complex scenarios. Regardless, CLAM opens up a noteworthy new direction for research on LLM long-term memory.
As agent systems' demand for long-term memory grows, how to achieve instantaneous consolidation in weight space without changing the model serving pipeline will become an important research topic. CLAM's attempt suggests that the fusion of parameterized memory and episodic memory may be a key step toward more persistent and more adaptive LLMs.
One-line Summary
CLAM compiles conversational activation streams into low-rank adapters on the fly via a hypernetwork, achieving gradient-free long-term memory consolidation for LLMs.
Paragraph Summary
CLAM proposes a dual-channel LLM long-term memory solution that fuses parameterized consolidation with an episodic buffer. Its recurrent hypernetwork compiles session activation streams into a low-rank adapter in a single forward pass, enabling instantaneous model weight updates without gradient steps or generation calls; the episodic buffer provides verbatim details at inference time. A gated latent state carries knowledge across sessions at a fixed size, avoiding model inflation. On the LoCoMo and LongMemEval-S benchmarks, CLAM leads agent baselines by 1-45 percentage points and gradient-based methods by 10-45 percentage points, while reducing memory construction time by 3-151 times. Ablation experiments show that parameterized consolidation and episodic memory are complementary, offering an efficient new paradigm for agent long-term memory.
Related articles

OpenAI to Shut Down Agent Builder and Two Other Services on November 30, Developers Must Migrate Urgently
OpenAI will shut down Agent Builder, the evals dashboard, and reusable prompts on November 30; evals is already read-only. Here's the scope, OpenAI's migration paths (Agent SDK, PromptFu, your own code), and what developers should export and rebuild this week.

RankSpot: A Pay-Per-Use SEO & AI Search Execution Layer That Lets Agents Write, Publish, and Build Backlinks for You
RankSpot is an SEO & AI search execution layer for AI Agents. Connect via MCP to Claude or ChatGPT to write, publish & build backlinks. Pay-per-use, $3/article.

Appxiom: Giving AI Coding Assistants Production Context
Appxiom gives AI coding agents production context—crashes, jank, memory leaks, failed network calls—across Android, iOS, Flutter, KMP, and Web, ranking bugs by Goal Friction Impact.