MDSKILL: Multi-View Distillation Truly Internalizes Agent Skill Memory

Self-evolving agents are one of the hottest directions in current AI research. Such agents accumulate experience by interacting with the environment and use that experience to continuously improve their own policies. However, efficiently "internalizing" past experience into model parameters—rather than relying solely on external prompts—has always been a challenge. A recent paper titled "MDSKILL: Learning to Internalize Skill Memory through Multi-View Distillation for Efficient Dynamic Agent Self-Evolution" proposes a multi-view distillation approach that attempts to address the noise and entanglement problems in the skill memory internalization process.

Pain Points of Skill Memory Internalization: Noise, Redundancy, and Conflict

Skill memory is typically stored as textual prompts capturing past experience, used to enhance reinforcement learning (RL) training. But the paper points out that such self-generated memory often contains noisy supervision signals, and the knowledge is functionally entangled, redundant, and even conflicting. Under a limited training budget, directly distilling such memory not only struggles to achieve knowledge internalization but may also cause performance degradation.

This observation hits a key bottleneck in current agent self-evolution pipelines. The common past approach was to concatenate skill memory into the context and rely on prompts to guide behavior, but this offers limited help for internal policy updates. While On-Policy Self-Distillation (OPSD) can provide dense, fine-grained supervision signals, it is highly sensitive to memory quality—which is exactly the entry point MDSKILL attempts to address.

On-Policy Self-Distillation (OPSD) is a self-distillation variant: the student model and teacher model are the same model, and the teacher signal comes from the model's "better" behavior on trajectories sampled from its own current policy (e.g., outputs from different reasoning branches or different views). Since the supervision signal comes from the model itself and provides token-level or step-level dense feedback on the same on-policy trajectory, OPSD offers finer granularity than sparse RL reward signals and stays closer to the model's current distribution. The cost is that it depends heavily on the quality of the "teacher signal"—if the self-generated skill memory itself is noisy or self-contradictory, distillation will solidify erroneous knowledge into the parameters and may even overwrite originally correct policies. This is precisely why the paper emphasizes the need for structured cleaning and decomposition of memory first.

Three-View Decomposition: Untangling Entangled Skill Memory

The core idea of MDSKILL is to decompose skill memory by function into three complementary views:

• State grounding: helping the agent understand the current environmental state;
• Action planning: guiding what action to take next;
• Execution control: constraining how actions are executed and their details.

On top of this decomposition, MDSKILL uses dynamic metrics to filter high-quality skills and constructs view-specific reasoning branches plus an integrated reasoning branch. OPSD then transfers and consolidates complementary experiential knowledge across these branches, aggregating it into a shared policy. In this way, knowledge from different views is no longer mixed-together noise but becomes structured experience that can be distilled separately and then merged.

According to the paper, this design enables the agent to complete skill internalization and continuous self-evolution without a stronger external teacher model. In other words, the distillation supervision signal comes from the agent's own performance across different views rather than relying on additional guidance from a larger model.

Skill memory typically refers to reusable experience entries accumulated by an agent over multiple task episodes, stored in natural language form—for example, "check whether the target object is in the fridge when you see one" or "filter first and then compare prices on a shopping site." Unlike ordinary in-context examples, skill memory is an external knowledge base that is continuously written to and retrieved across episodes and tasks; common implementations include vector-retrieval experience pools or prompt libraries organized by task type. Its advantage is that it can be updated without training; its disadvantage is that entries are self-generated by the model, vary in quality, and may overlap or even contradict each other. As entries grow, so does the length and noise of what gets concatenated into context. MDSKILL's starting point is precisely this: rather than continuously stuffing memory into context, distill that memory into policy parameters so the model "remembers" instead of "looks it up."

Experimental Results: Gains on ALFWorld and WebShop

The paper evaluates MDSKILL on three agent benchmarks, comparing against pure RL methods, skill-augmented methods, and self-distillation baselines that do not rely on a stronger external teacher model. Results show that without additional computational overhead, MDSKILL improves SOTA performance by up to 7.1% and 8.1% on ALFWorld and WebShop, respectively.

These two benchmarks correspond to embodied household tasks and online shopping environments, both of which place high demands on state understanding, multi-step planning, and execution details—precisely matching MDSKILL's three-view design. The paper further validates the method's effectiveness in knowledge internalization and policy improvement through extensive ablation experiments and evolution analyses. The code has been released in an anonymous repository (anonymous.4open.science/r/MDSKILL-CF53) for the community to reproduce and verify.

ALFWorld is a text-based benchmark for embodied household tasks: the agent moves through a simulated residence following text instructions, searches containers, and manipulates objects (e.g., "put a clean cup in the microwave"), requiring long-horizon multi-step planning and state tracking. WebShop simulates an online shopping environment where the agent must search product pages, filter attributes, compare prices, and complete purchases based on user needs, testing fine-grained instruction understanding and action-sequence control. Both use task success rate as the primary metric, and both feature discrete action spaces and long episodes—mapping respectively to the combined pressure of "state grounding + execution control" and "action planning + execution control." They are therefore often used as touchstones for whether skill-memory and planning methods are truly internalized.

Why It Matters: An Efficiency Path for Self-Evolution

The significance of MDSKILL goes beyond the numerical gains on two benchmarks. It proposes a "lighter" self-evolution path: no reliance on a stronger external teacher, no additional compute, but improved distillation efficiency through reorganizing the structure of existing skill memory. For teams looking to deploy self-evolving agents under limited training budgets, this approach offers direct engineering reference value.

Of course, whether the three-view decomposition applies to more task types and how stable the dynamic filtering metrics are in more complex environments remain to be verified by future work. But based on the currently public experimental results, MDSKILL provides a clearly structured, reproducible framework for skill memory internalization.

One-line Summary

MDSKILL decomposes skill memory into three views and applies multi-view distillation to achieve efficient agent self-evolution without an external teacher.

Paragraph Summary

The paper proposes MDSKILL to address the difficulty of internalizing skill memory in self-evolving agents. The authors point out that self-generated skill memory stored as textual prompts contains noisy supervision signals, and the knowledge is functionally entangled, redundant, and even conflicting; under a limited training budget, direct distillation not only struggles to internalize but may degrade performance. MDSKILL decomposes skill memory by function into three complementary views—state grounding, action planning, and execution control—uses dynamic metrics to filter high-quality skills, constructs view-specific reasoning branches and an integrated reasoning branch, and then uses On-Policy Self-Distillation to transfer complementary knowledge across branches and aggregate it into a shared policy. The method does not rely on a stronger external teacher model and adds no extra computational overhead, improving SOTA performance by up to 7.1% and 8.1% on ALFWorld and WebShop, respectively; ablations and evolution analyses validate its effectiveness.