VLM Becomes a Joint Denoiser: JADE Makes Robot Action Generation More Precise

Diffusion-based vision-language-action (VLA) models typically cascade a pretrained vision-language model (VLM) with a diffusion action expert, but the VLM often plays only the role of a static condition provider. This separation limits how deeply pretrained perception and reasoning capabilities can guide action generation, especially in out-of-distribution scenarios. A new study proposes JADE (Joint Action DEnoiser), which lets the VLM participate alongside the action expert in refining actions, opening up new possibilities for robot control.

From Static Conditions to Dynamic Guidance: A Fundamental Shift in the VLM's Role

In traditional VLA architectures, the VLM understands visual observations and language instructions and outputs a static context representation, based on which the diffusion action expert generates actions. The problem is that this one-way information flow prevents the VLM's reasoning ability from dynamically adjusting to the intermediate states of action generation. JADE's core insight is this: if the action latent has a structured form, the VLM can read the current noisy action latent just as it reads visual and language inputs.

This design allows the VLM to contextualize the noisy action together with multimodal inputs at every denoising step, and to guide the action expert through a gated cross-attention mechanism. In other words, the VLM's guidance signal evolves as the action is progressively generated, rather than being fixed before generation begins.

Diffusion models and the denoising process: Diffusion models generate samples by gradually adding noise to data and then learning to reverse the denoising. In robot action generation, the action sequence (or action latent) starts from pure noise and, through multiple denoising steps, gradually becomes an executable action trajectory. Each denoising step relies on the current noisy action and condition information to predict the noise or the clean action. In traditional VLA, the condition information (VLM output) is fixed before denoising, whereas JADE lets the VLM re-read the noisy action latent at every denoising step, achieving dynamic conditioning.

Gated cross-attention mechanism: Cross-attention allows the action expert to query the VLM's multimodal representation, while gating dynamically controls the strength of information inflow—like a "valve" that adjusts the weight of the VLM's guidance signal according to the current noisy action state. This enables the VLM to judge whether the current action state needs more language or visual reasoning involvement, avoiding the blindness of static conditions.

Joint Denoising Training Mechanism and Efficiency Optimization

JADE not only lets the VLM participate in guidance but further supervises the VLM to recover the clean action latent from the noisy action latent. This training objective provides the VLM with a direct action-learning signal, tightly coupling perception and control. According to the paper, this joint denoising approach brings an additional benefit: JADE can generate actions with fewer sampling steps, thereby partially offsetting the extra computational cost brought by the VLM's participation.

The research team also introduces an adaptive step-size scheduling strategy to balance task performance and inference latency. This means the system can dynamically adjust computational investment according to actual needs, rather than always using the same sampling configuration.

The π₀.₅ baseline model: π₀.₅ is a vision-language-action foundation model proposed by Physical Intelligence, adopting an architecture that cascades a pretrained VLM with a diffusion action expert, and performs strongly on robotic manipulation tasks. JADE uses π₀.₅ as its base, retaining its pretrained VLM and action expert but modifying how the two interact, so that the VLM participates in the denoising loop. Therefore, the comparison between JADE and π₀.₅ directly reflects the gain of "dynamic guidance" over "static conditions."

Adaptive step-size scheduling: Diffusion sampling step size determines the granularity of the denoising trajectory. A fixed step size makes it hard to balance quality and speed: too large a step easily loses details, while too small a step incurs high latency. Adaptive scheduling dynamically adjusts the step size according to the current denoising state (e.g., large steps early to approach quickly, small steps later for fine adjustment), allocating computational resources on demand at inference time—a key engineering technique by which JADE mitigates the extra overhead brought by the VLM's participation.

Benchmark and Real-Robot Experimental Performance

On the RoboDojo benchmark, JADE achieves an average success rate of 12.67%, ranking highest among all evaluation methods based on π₀.₅ and being competitive with recent large-scale pretrained VLA and WAM foundation models. On RoboTwin 2.0, JADE likewise brings performance improvements.

More noteworthy are the real-robot experiments: across four real-world dual-arm manipulation tasks, JADE improves the average success rate by 20.8 percentage points compared with π₀.₅. This margin is considerable in the field of robotic manipulation, indicating that the joint denoising mechanism is equally effective in real physical environments.

RoboDojo and RoboTwin 2.0: RoboDojo is a simulation benchmark for robotic manipulation, covering a variety of tasks and scenarios, used to evaluate the generalization and success rate of VLA models; RoboTwin 2.0 is a dual-arm robot simulation platform that emphasizes coordinated manipulation. Both are often used to test model performance on in-distribution and out-of-distribution tasks. JADE achieves a 12.67% average success rate on RoboDojo; although the absolute value is not high, it is already a competitive level in a highly challenging benchmark, and it ranks highest among all methods based on π₀.₅.

Implications for VLA Architecture Design

The significance of JADE goes beyond a stronger model. It challenges an implicit default assumption in the VLA field—that the VLM should only serve as a static feature extractor. By having the VLM participate deeply in the denoising loop, the rich perception and reasoning capabilities contained in the pretrained model can continuously act throughout the entire action generation process.

This design approach also offers new possibilities for out-of-distribution generalization: when the VLM can "see" what state the action is currently in, it is more likely to use pretrained knowledge to correct generation trajectories that deviate from the norm. Of course, the VLM's participation in every denoising step inevitably increases computational burden. JADE alleviates this problem by reducing sampling steps and adaptive scheduling, but the efficiency trade-offs in actual deployment still require further validation.

From a broader perspective, JADE represents a trend of VLA models moving from "cascading" toward "deep fusion." As the boundaries between perception, language, and control become further blurred, robot systems are expected to exhibit stronger adaptability in open environments.

One-line Summary

JADE lets the VLM participate in the diffusion denoising loop to dynamically guide action generation, improving real-robot success rate by 20.8 percentage points.

Paragraph Summary

Targeting the problem that in traditional VLA models the VLM acts only as a static condition provider and cannot dynamically guide action generation, JADE (Joint Action DEnoiser) proposes letting the VLM read the noisy action latent at every denoising step and dynamically guide the action expert through gated cross-attention. In training, JADE supervises the VLM to recover the clean action latent from noise, giving it a direct action-learning signal, while supporting fewer sampling steps and, together with adaptive step-size scheduling, balancing performance and latency. On the RoboDojo benchmark, JADE achieves a 12.67% average success rate, the highest among methods based on π₀.₅; across four real-world dual-arm manipulation tasks, it improves the average success rate by 20.8 percentage points compared with π₀.₅. This work challenges the default assumption that the VLM serves only as a static feature extractor, representing a trend of VLA architectures moving from cascading toward deep fusion.