[KongchangAI]
· 2 min read· 1,297 words

RePlan-VLA: Progress-Aware Replanning Breakthrough for Long-Horizon Robotic Manipulation

RePlan-VLA: Progress-Aware Replanning Breakthrough for Long-Horizon Robotic Manipulation

RePlan-VLA decouples long-horizon responsibilities via an external execution-aware framework, boosting VLA success on long-sequence tasks.

RePlan-VLA is a VLA-agnostic external execution-aware framework that addresses VLA models' performance degradation on long-horizon multi-step tasks. Instead of burdening the VLA policy with long-horizon reasoning, memory, and error recovery, it focuses on short-horizon action generation while an independent framework monitors end-effector motion to evaluate progress and adaptively replan. It requires no fine-tuning of the underlying policy. On RoboCasa365 and CALVIN, it improves success rates by 21.9% and 10.4%, respectively, and was validated on a Franka Research 3 robot.

Vision-Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robotic manipulation, but their performance often degrades significantly on long-horizon tasks that require multiple interdependent steps. The traditional approach asks VLA policies to simultaneously handle long-horizon reasoning, memory, action generation, progress tracking, and error recovery—multiple responsibilities that only increase the model's burden. The recently proposed RePlan-VLA framework takes a different path: it does not modify the underlying VLA policy, but instead uses an external execution-aware framework to convert the VLA's short-horizon manipulation capabilities into reliable long-horizon execution.

RePlan-VLA paper title screenshot

The Challenge of Long-Horizon Tasks and RePlan-VLA's Breakthrough Approach

Long-horizon tasks typically involve multiple interdependent manipulation steps, such as the sequence "open the drawer, take out the object, close the drawer." Although VLA models perform well on single-step operations, they are prone to failure on long sequences due to error accumulation and difficulty tracking state. The core insight of RePlan-VLA is: rather than making the VLA model responsible for long-horizon reasoning—something it is not good at—let it focus on short-horizon action generation, while delegating progress monitoring and replanning to an independent execution-aware framework.

The framework is VLA-agnostic, meaning it can be paired with a variety of existing VLA policies without additional fine-tuning of the underlying policy. This greatly lowers the deployment barrier and preserves the original model's capabilities.

VLA models (Vision-Language-Action Models) are a class of robot control models that unify visual perception, language instructions, and action generation into a single neural network. Their typical architecture uses a pretrained vision-language model (such as CLIP, LLaVA, etc.) as the backbone, with an added action decoding head or adapter layer that maps multimodal representations into control commands for the robot's end-effector. The advantage of VLA models is that they can directly understand natural language instructions and generalize to new objects and scenes, but their training data mostly consists of short-horizon manipulation clips, leaving the models without explicit memory and planning mechanisms for long-sequence tasks. In addition, VLA models are typically trained via imitation learning and generate actions step by step at inference time, so errors gradually accumulate over long sequences. RePlan-VLA targets exactly this structural limitation by stripping long-horizon responsibilities away from the VLA policy.

Progress-Aware and Adaptive Replanning Mechanism

RePlan-VLA evaluates task progress by explicitly monitoring the robot's motion. Specifically, it continuously observes the movement trajectory and state changes of the robot's end-effector to determine whether the current subtask has been completed or whether an anomaly has occurred. Based on these observations, the framework adaptively revises future plans—if it detects that a step has not achieved the expected effect, it replans subsequent actions instead of mechanically executing the original sequence.

This mechanism resembles the "executive monitoring" humans perform when completing complex tasks: we constantly check whether the current step has succeeded and adjust subsequent actions accordingly. RePlan-VLA externalizes this capability, avoiding excessive cognitive load on the VLA model itself.

End-effector refers to the device at the end of a robotic arm used to interact with the environment, commonly including parallel grippers, suction cups, or multi-fingered dexterous hands. In long-horizon manipulation, the end-effector's pose, open/close state, and motion trajectory are key signals for judging whether a subtask has been completed. For example, the success of the "grasp the cup" subtask can be inferred from whether the relative pose between the end-effector and the cup remains stable after the gripper closes. RePlan-VLA builds an explicit estimate of task progress by continuously tracking the end-effector's motion trajectory and state changes, without relying on implicit representations inside the VLA model. This motion-signal-based monitoring approach has low computational overhead and is decoupled from the specific VLA model, which is the technical basis for its VLA-agnostic property.

Experimental Validation: Benchmarks and Real-Robot Deployment

On two long-horizon manipulation benchmarks, RoboCasa365 and CALVIN, RePlan-VLA improved success rates by 21.9% and 10.4%, respectively, compared with the strongest prior baseline methods. These two benchmarks are specifically designed to evaluate multi-step, long-sequence robotic manipulation capabilities, and the magnitude of improvement indicates that the framework effectively mitigates performance degradation on long-horizon tasks.

More importantly, the research team validated the feasibility of real-world deployment on a Franka Research 3 robot. Although the paper does not disclose specific success-rate data in real environments, this validation demonstrates that the framework is not only applicable to simulated environments but also has potential for practical robotic applications.

RoboCasa365 and CALVIN are two widely used long-horizon benchmarks in the field of robotic manipulation. CALVIN (Composing Actions from Language and Vision) requires agents to complete consecutive multi-step subtasks based on language instructions, emphasizing language grounding and long-sequence composition ability, with evaluation metrics including the average number of completed subtasks and task success rate. RoboCasa365 is based on a large-scale simulated environment and covers hundreds of long-horizon manipulation tasks in scenarios such as kitchens and homes, focusing on evaluating model generalization across diverse objects and scenes. The common feature of these two benchmarks is long task sequences with dependencies among subtasks, which correspond exactly to the scenarios where VLA models suffer the most severe performance degradation. RePlan-VLA's improvements on these two benchmarks (21.9% and 10.4%) reflect its gains in compositional instruction following and diverse long-sequence manipulation, respectively.

Implications for the VLA Ecosystem

The value of RePlan-VLA lies not only in performance improvement but also in its modular approach: decoupling different responsibilities in long-horizon execution. The VLA model handles what it does best—short-horizon action generation—while progress evaluation and replanning are handled by independent modules. This design enables existing VLA models to gain enhanced long-horizon capabilities without retraining, which, for the rapidly iterating field of robot learning, means lower transfer costs and more flexible deployment options.

In the future, this execution-aware framework may be combined with more types of VLA models and even extended to multi-robot collaboration or long-horizon tasks in dynamic environments. However, the current framework's reliance on motion monitoring also raises a question: in highly unstructured environments or those with significant perception noise, will the accuracy of progress evaluation become a new bottleneck? This remains to be answered by future research.

Franka Research 3 is a seven-degree-of-freedom collaborative robotic arm launched by Franka Emika, widely used in robot learning research. Its joint torque sensors and compliant control capabilities make it suitable for fine manipulation tasks, and it provides interfaces to simulation environments such as ROS and PyBullet, facilitating sim-to-real transfer validation. In the RePlan-VLA study, Franka Research 3 was used to validate the framework's feasibility in real physical environments, including the real-time performance of end-effector motion monitoring and the robustness of the replanning mechanism under real perception noise. Choosing this platform also means the framework's design took into account hardware conditions commonly available in the research community, which is conducive to subsequent reproduction and adoption.

One-line Summary

RePlan-VLA decouples long-horizon responsibilities via an external execution-aware framework, significantly improving VLA models' success rates on long-sequence manipulation.

Paragraph Summary

RePlan-VLA is an external execution-aware framework that is VLA-agnostic and aims to solve the significant performance degradation of VLA models on long-horizon multi-step tasks. Its core idea is not to burden the VLA policy with long-horizon reasoning, memory, and error recovery, but to let it focus on short-horizon action generation, while an independent framework evaluates task progress by explicitly monitoring the robot end-effector's motion trajectory and state changes and adaptively revises future plans. The framework requires no fine-tuning of the underlying VLA policy, lowering the deployment barrier. On the two long-horizon manipulation benchmarks RoboCasa365 and CALVIN, RePlan-VLA improves success rates by 21.9% and 10.4%, respectively, over the strongest baselines, and its real-world deployment feasibility was validated on a Franka Research 3 robot. This modular decoupling approach offers a low-cost way to enhance long-horizon capabilities for the VLA ecosystem, but it also raises new questions about the accuracy of progress evaluation in unstructured environments.

#RePlan-VLA#VLA models#long-horizon robotic manipulation#progress-aware replanning#RoboCasa365#CALVIN benchmark#Franka Research 3
Share:

Related articles