LoopWAM: Enabling Robots to Evolve in a Closed Loop from Deployment Failures

Title

LoopWAM: Enabling Robots to Evolve in a Closed Loop from Deployment Failures

Meta Description

LoopWAM introduces a closed-loop world-action model that learns from both successful and failed data in deployment rollouts, raising success rates from 23.67% to 43.33% over three iterations on RoboChallenge Table30-V2, and reaching 94.5% on RoboTwin 2.0.

Article Content

The robotics field has long sidestepped a difficult problem: no matter how much expert demonstration data you have, it cannot make a general-purpose policy improve itself during real-world deployment. The reason is straightforward—real rollouts are a mix of successes, mediocre operations, and failures. If you imitate everything indiscriminately, you end up learning bad behaviors too. If you cherry-pick only successful samples, you throw away the most valuable failure evidence from physical interaction. LoopWAM attempts to tackle this problem head-on.

Core Idea: Don't Filter Quality—Learn to Exploit It

LoopWAM is a closed-loop world-action model, and the key is that it doesn't avoid low-quality data but instead learns from the full quality spectrum. Training data comes from two channels: the model's own deployment rollouts, and rollouts produced by other deployed policies.

Several technical designs are worth noting here.

Source-Dependent Noise Sampling

Through source-dependent noise sampling, LoopWAM unifies behavior learning and action-conditioned world modeling into a single framework. Simply put, the model simultaneously learns "what to do" and "what happens after you do it," rather than splitting these into separate modules.

Source-dependent noise sampling is motivated by the fact that rollouts from different sources carry different information density and reliability. Data from the model's own deployment and data from other policies differ systematically in action distribution, failure modes, and environment coverage. If the same noise or diffusion strength is applied to all data, the model may over-trust one source, or fail to distinguish "failing because the policy is bad" from "failing because the task itself is hard." This mechanism dynamically adjusts noise levels or sampling weights based on data source, keeping behavior cloning signals and world modeling signals balanced during training. This contrasts with the fixed noise schedules commonly used in diffusion policies, and is closer to a meta-learning-style data confidence estimation, allowing mixed-quality data to each play its proper role within a unified framework.

Quality Steering and Quality-Weighted Supervision

This is the core mechanism by which LoopWAM handles mixed-quality data. Quality steering allows the model to retain complete supervision signals from videos of all quality levels, while quality-weighted supervision modulates action imitation—successful samples receive higher imitation weights, and failed samples are used more for understanding physical consequences rather than being directly discarded.

The distinction between quality steering and quality-weighted supervision is worth clarifying. The former operates at the video/observation level, retaining complete supervision over rollouts of all quality levels, meaning the model must learn to predict the evolution of subsequent frames even when viewing failed trajectories, thereby grasping physical laws; the latter operates at the action level, reweighting the imitation loss: successful segments receive higher action cloning weights, while failed segments have their proportion in the action loss reduced, but their contribution to the world modeling loss is retained. This strategy of "treating observations and actions separately" avoids both the information loss caused by simply filtering out failed samples and the behavioral degradation caused by indiscriminate imitation. Essentially, it decouples "what to learn" from "how deeply to learn it," and is a key design for handling long-tail failures in real deployment data.

Parallel Denoising of Video and Action Tokens

During the parallel denoising process, video tokens and action tokens interact throughout. This means the "consequences" the model learns can directly influence action generation, rather than being a post-processing layer applied after action generation. This coupling approach makes closed-loop improvement possible.

World-Action Model is a class of unified architecture that has emerged in robot learning in recent years, with a core assumption: action decisions and predictions of environment dynamics should not be split into two independent modules. Traditional approaches typically train policy learning (what to do) and world models (what happens after you do it) separately, then stitch them together at inference time; world-action models instead let the two share representations and co-optimize within the same denoising or generation process. The benefit is that the model can directly "rehearse" the physical consequences of its actions when generating them, enabling more reasonable responses to variables like contact forces and object displacements that are difficult to infer from single-frame observations. LoopWAM is built on exactly this paradigm and further extends it to closed-loop deployment scenarios: not only using world modeling to assist actions, but also using consequences observed in real rollouts to inversely correct the action distribution.

Experimental Results: Three Rounds of Deployment-Driven Improvement of Nearly 20 Percentage Points

LoopWAM was validated on three benchmarks, with the most compelling being the closed-loop experiments on RoboChallenge Table30-V2.

After three "deploy-train" cycles, the success rate rose from 23.67% to 43.33%, an increase of 19.7 percentage points, with a final score of 56.80. The significance of this number isn't in how high the absolute value is, but in proving that deployment rollouts can indeed be converted into sustained improvements in policy capability—which is precisely the core demand of closed-loop improvement.

On RoboTwin 2.0, LoopWAM achieved an average success rate of 94.5%; on RoboCasa365, an overall success rate of 49.5%. An important caveat: these results were obtained without additional embodied pre-training, and the comprehensive performance across all three benchmarks was the best among compared methods.

Single-task post-training scenarios are equally compelling. On a high-difficulty task like shirt folding, after single-task post-training followed by two rounds of deployment-driven improvement, the success rate rose from 13.20% to 55.56%. The leap from 13% to 55% demonstrates that the closed-loop mechanism is effective not only for multi-task general-purpose policies, but can also extract considerable improvement headroom in single-task fine-tuning scenarios.

Why This Direction Matters

Scaling robot policies has always faced an awkward reality: data scale goes up, but model performance in real environments doesn't automatically improve. LoopWAM's approach is to treat deployment itself as a source of training signals, rather than as an "acceptance check" after training is complete.

The significance of this shift is that it makes failures no longer noise to be filtered out, but necessary information for understanding how the physical world works. Every operation a robot botches in a real environment carries specific knowledge about contact forces, object dynamics, and task boundaries—things that pure successful demonstration data struggles to cover.

Of course, the engineering complexity of closed-loop improvement is considerable: it requires a stable deployment pipeline, a reliable rollout collection mechanism, and a training framework capable of handling mixed-quality data. LoopWAM provides a workable solution at the methodological level, and whether it can be reproduced across more task types and hardware platforms is a direction worth tracking.

One-line Summary

LoopWAM uses a closed-loop world-action model to continuously improve robot policies from mixed-quality deployment data.

Paragraph Summary

LoopWAM introduces a closed-loop world-action model that doesn't filter deployment rollout quality, but instead unifies behavior learning and action-conditioned world modeling through source-dependent noise sampling, quality steering, and quality-weighted supervision. Video and action tokens interact throughout parallel denoising, allowing predicted consequences to directly influence action generation. On RoboChallenge Table30-V2, three deploy-train cycles raised the success rate from 23.67% to 43.33%, a 19.7 percentage point gain; RoboTwin 2.0 reached 94.5%, and RoboCasa365 reached 49.5%, all without additional embodied pre-training. Single-task post-training on shirt folding improved the success rate from 13.20% to 55.56% over two deployment-driven cycles. This method treats deployment itself as a training signal, making failure necessary information for understanding the physical world.