[KongchangAI]
· 2 min read· 1,134 words

PMR: Customizing Memory Rubrics for Each User to Make AI Agents Understand You Better

PMR: Customizing Memory Rubrics for Each User to Make AI Agents Understand You Better

PMR framework achieves zero inference overhead memory update policy learning through two-layer personalized rubrics and token-level credit assignment.

This article addresses the challenge of learning memory update policies for long-term companion AI agents, noting that traditional methods rely on terminal QA accuracy leading to coarse feedback and neglect user heterogeneity. Researchers propose the Personalized Memory Rubrics (PMR) framework, which decomposes evaluation into a general layer and a user-adaptive layer and uses rubric-guided token-level credit assignment to amplify verified decisions on successful trajectories while falling back to standard optimization on failures. Rubrics and memory policies co-evolve, and deployment requires no external judges, achieving zero inference overhead. Experiments across four benchmarks show PMR consistently outperforms strong memory-augmented baselines.

Long-term companion AI agents need to maintain evolving memories, continuously deciding which information to add, modify, or discard. However, learning optimal memory update policies is not easy. Traditional training methods rely solely on final question-answering accuracy as an end signal, resulting in overly coarse feedback where all intermediate updates are treated equally. More critically, the utility of memory updates varies from person to person—the same memory operation may have completely opposite effects for different users. Existing methods fail to address this heterogeneity, relying on globally shared evaluation criteria and providing at most a single scalar intermediate reward.

PMR paper title screenshot

To address these challenges, researchers propose Personalized Memory Rubrics (PMR), a framework that transforms terminal feedback into fine-grained, step-by-step, and user-specific credit assignment. PMR decomposes evaluation into two layers: a general layer that captures failure patterns across user types, and a per-user adaptive layer that models individual characteristics as specialized increments over the general baseline. Through Rubric-Guided Credit Assignment, PMR redistributes credit at the token level, selectively amplifying verified decisions on successful trajectories while falling back to standard policy optimization on failed trajectories to avoid unreliable attribution. Rubrics and memory policies co-evolve in a closed loop with anti-circularity guarantees. Crucially, PMR requires no rubrics or external judges at deployment time, achieving zero inference overhead. Extensive experiments across four benchmarks show that the method achieves consistent performance improvements over strong memory-augmented baselines.

Why Memory Update Policies Are Hard to Learn

The core challenge of long-term personalized agents lies in dynamic memory management. Agents need to determine which information is worth retaining, which needs correction, and which should be deleted. Traditional methods use final question-answering accuracy as the sole training signal, meaning that regardless of intermediate memory update quality, all updates receive the same credit assignment. This coarse-grained feedback cannot guide agents to finely adjust each memory operation.

The deeper problem is user heterogeneity. The same memory operation may have opposite effects for different users—a memory addition that works for one user may cause interference for another. Existing methods adopt globally shared evaluation criteria and provide at most a single scalar intermediate reward, unable to capture such inter-user differences. PMR is designed precisely to address these two pain points.

Credit Assignment is a core challenge in reinforcement learning: when an agent executes a sequence of actions and receives a final reward, how to determine which actions in the sequence contributed to positive or negative outcomes. In long-term memory agents, a single conversation may involve dozens of memory additions, modifications, or deletions, yet only question-answering accuracy serves as the final signal, making it difficult to locate good versus bad operations. Traditional methods like REINFORCE or PPO perform credit assignment through policy gradients but rely on scalar rewards with high variance. PMR's "token-level credit assignment" draws on fine-grained attribution ideas, decomposing terminal feedback onto each memory update token to achieve more precise policy optimization.

PMR's Core Design: Two-Layer Evaluation and Token-Level Credit Assignment

PMR decomposes evaluation into a general layer and a user-adaptive layer. The general layer captures common failure patterns across user types, while the user-adaptive layer models individual traits as increments over the general baseline. This design allows evaluation to leverage group-level regularities while retaining sensitivity to individual differences.

In the credit assignment phase, PMR employs rubric-guided token-level redistribution. For successful trajectories, the system selectively amplifies verified decision signals; for failed trajectories, it falls back to standard policy optimization, preventing unreliable attribution from affecting training. This selective credit assignment mechanism enables agents to learn more from successes while not being misled by noise in failed trajectories.

Rubric in the evaluation domain refers to a structured set of scoring rules used to decompose complex outputs into multiple evaluable dimensions. The rubrics in PMR are not manually set but co-evolve with memory policies: general-layer rubrics capture common failure patterns across users (such as factual errors or redundant information), while user-adaptive layers learn individual preferences from a small amount of user interaction data (such as sensitivity to privacy or preference for detail). Together, the two layers of rubrics generate fine-grained feedback for each memory operation, which is then converted into policy gradient signals through token-level credit assignment. This design avoids the bias of manually designed rubrics while preserving interpretability.

Zero Inference Overhead Deployment Advantage

A notable advantage of PMR is that no rubrics or external judges are needed at deployment. This means the system incurs no additional overhead during actual inference. For personalized agent scenarios requiring real-time responses, this characteristic is crucial. Rubrics and memory policies co-evolve during training with anti-circularity guarantees, ensuring that the two do not fall into mutually reinforcing error loops.

Anti-circularity guarantee refers to preventing rubrics and policies from mutually reinforcing errors and entering self-loops during co-evolution. For example, if rubrics always give high scores to the policy's current behavior, the policy tends to repeat that behavior, causing evaluation to lose objectivity. PMR introduces independent validation signals (such as calibration based on real user feedback) and regularization constraints to ensure that rubric updates do not depend on the policy's current outputs, thereby maintaining evaluation reliability. This mechanism is especially important for deployment scenarios without external judges, where circularity risk is higher due to the lack of independent supervision.

Experimental Validation and Results

Researchers conducted extensive experiments on four benchmarks, and results show that PMR achieves consistent performance improvements over strong memory-augmented baselines. This demonstrates that fine-grained, user-specific credit assignment can indeed improve the learning effectiveness of memory update policies, validating the framework's design.

Implications for Personalized AI Agents

PMR's approach reveals an important direction: memory management should not be treated as a globally uniform process but requires personalized adjustment based on user characteristics. As AI agents increasingly take on roles such as long-term companions and personalized assistants, how to enable agents to learn "what to remember for whom" will become a key capability. PMR's two-layer evaluation framework and token-level credit assignment mechanism provide a technical path worth drawing on for this direction.

One-line Summary

PMR framework achieves zero inference overhead memory update policy learning through two-layer personalized rubrics and token-level credit assignment.

Paragraph Summary

This article addresses the challenge of learning memory update policies for long-term companion AI agents, pointing out that traditional methods rely on terminal question-answering accuracy leading to coarse feedback and neglect user heterogeneity. To this end, researchers propose the Personalized Memory Rubrics (PMR) framework, which decomposes evaluation into a general layer and a user-adaptive layer, and through rubric-guided token-level credit assignment, amplifies verified decisions on successful trajectories while falling back to standard optimization on failed trajectories. Rubrics and memory policies co-evolve, and deployment requires no external judges, achieving zero inference overhead. Experiments across four benchmarks show that PMR consistently outperforms strong memory-augmented baselines, providing new ideas for memory management in personalized AI agents.

#Personalized Memory Rubrics#PMR framework#AI agent memory#token-level credit assignment#personalized AI agents
Share:

Related articles