LIBERO-RIG: How Robot Hardware Perturbations Break VLA Model Robustness

When Robot Hardware Goes Off-Script, How Long Can VLA Models Hold Up?

The deployment environment for robot policies is never an ideal laboratory. Cameras get bumped and remounted, calibration parameters drift over time, controllers run at inconsistent frequencies, end-effector pads gradually wear down—these hardware-level changes are nearly unavoidable in everyday deployment. Yet current evaluation frameworks for vision-language-action (VLA) models focus almost entirely on visual generalization, language instruction generalization, or task generalization, with no systematic quantification of robustness to changes in the robot's own hardware state.

LIBERO-RIG is a benchmark designed specifically to fill this gap. Its fundamental difference from existing work is that it only perturbs the robot's hardware and setup, keeping the LIBERO tasks themselves completely unchanged. This "controlled variable" design allows researchers to cleanly isolate the independent effects of hardware factors on policy performance.

A VLA (vision-language-action) model is a type of policy model that unifies visual perception, natural language instruction understanding, and robot action generation into a single end-to-end framework. Typical examples include RT-2, OpenVLA, and π0. The basic approach is to use a pretrained vision-language model (such as CLIP or PaLI) as a backbone, then map language and visual representations to continuous robot control commands through action tokenization or a diffusion head. Because these models inherit large-scale internet pretraining's semantic understanding capabilities, VLA models excel at task generalization across objects, scenes, and instructions. However, the training and evaluation of such models almost universally assume that the hardware platform itself is fixed: camera positions, controller parameters, and actuator response characteristics remain consistent between training and deployment. Once hardware undergoes physical drift, the "vision→action" mapping learned by the model develops systematic biases. LIBERO-RIG is precisely a stress test targeting this overlooked assumption.

Eleven Perturbation Axes and 210 Conditions

LIBERO-RIG covers eleven perturbation dimensions, each with five severity levels, ultimately combining into 210 test conditions across four LIBERO suites. These perturbation axes can be broadly categorized:

Perception-level perturbations include changes in camera mounting position and optical parameters, as well as calibration offsets between the camera and the robot arm. These perturbations directly alter the visual input the model receives.

Control and execution-level perturbations encompass drive gain, control frequency, proprioceptive noise, base pose, table height, and gripper pad state. These factors affect the quality of action generation and physical interaction, while the visual image may remain completely unchanged.

Each condition is evaluated using 500 episodes, covering nine recent VLA models, supplemented by rigorous statistical tests to verify the significance of each finding. This evaluation scale is considerable among comparable benchmarks.

Drive gain refers to the proportional coefficient by which the controller amplifies desired velocity or torque commands before sending them to the motors. Higher gain leads to action overshoot and oscillation, while lower gain causes sluggish response. Control frequency refers to the rate at which policy inference and the underlying control loop operate; inconsistent frequencies cause temporal misalignment between action interpolation and actual execution, which is particularly sensitive in tasks requiring fine force control. Proprioceptive noise refers to random errors in feedback signals such as joint encoders and torque sensors, which directly corrupt the model's state input. Gripper pad state alters the end effector's effective geometry and friction characteristics, shifting grasping force and contact position. Together, these factors constitute a "proprioception and control channel" perturbation set, characterized by the fact that camera images may remain completely unchanged while the robot's actual motion response has already deviated from the training distribution.

How Severe Is the Performance Drop?

The core data provides a clear answer. The aggregate success rate drops from a baseline of 0.932 to 0.767 at moderate severity, and further falls to 0.509 at the highest severity level. This means that even "moderate" hardware perturbations can cost approximately 18 percentage points in success rate; under extreme conditions, nearly half of all tasks fail.

More noteworthy is that the magnitude of performance degradation varies dramatically across different perturbation axes. Some hardware changes barely affect performance, while others cause catastrophic declines. This non-uniformity indicates that VLA models' robustness weaknesses are concentrated in specific dimensions rather than being uniformly fragile.

The choice of action interface also significantly affects degradation patterns. Different models exhibit vastly different failure characteristics when facing the same hardware perturbations, suggesting that model architecture and training strategy produce substantive differences in hardware robustness.

The action interface refers to the specific representation and execution method of the policy's output actions. Common types include joint position increments, end-effector pose increments (delta pose), and absolute joint angles. Different interfaces vary greatly in sensitivity to hardware perturbations: pose increment interfaces depend on precise forward kinematics and calibration parameters, and calibration offsets directly amplify their errors; joint position increment interfaces, while insensitive to visual calibration, are more susceptible to changes in drive gain and control frequency. Additionally, models using action chunking or diffusion generation exhibit different tolerance to control frequency perturbations in terms of output smoothness. Therefore, the observation in LIBERO-RIG that "different models have vastly different failure characteristics" can largely be attributed to differences in action interfaces and generation mechanisms, rather than purely visual robustness differences.

Why Visual Perturbation Training Can't Save Hardware Failures

A counterintuitive finding is that training on visual perturbations does not mitigate those hardware failures that do not alter camera images.

This conclusion poses a direct challenge to current data augmentation practices. Many teams use visual augmentation techniques such as random cropping, color jittering, and Gaussian blur to improve policy generalization. But LIBERO-RIG's results show that when perturbations occur in non-visual channels such as control frequency, drive gain, or calibration parameters, visual-level augmentation provides virtually no protection.

The logic behind this is not complicated: if the camera image is unchanged but the robot's actual motion response has already changed, a misalignment emerges between the "world" the model sees and the "actions" it can actually execute. VLA models trained purely on visual input lack the mechanism to perceive this misalignment.

Implications for VLA Research and Deployment

LIBERO-RIG's release includes a complete perturbation parameter manifest, ensuring that each episode's hardware conditions are reproducible. This infrastructure-level contribution may have more long-term value than the benchmark data itself—it transforms "hardware robustness" from a vague engineering concern into a quantifiable, comparable research problem.

For practical deployment, the conclusion is clear: VLA models' high success rates under laboratory conditions should not be directly extrapolated to real-world scenarios. A single collision with a camera mount, or a frequency shift caused by controller load fluctuations, can cause performance to plummet.

For the research community, LIBERO-RIG points to a long-overlooked direction: hardware-aware training and adaptation mechanisms. Models need not only to "see more accurately" but also to "perceive changes in their own bodies."

Hardware-aware training and adaptation mechanisms are the key direction identified by LIBERO-RIG, with the core idea being to enable policies to not only acquire information from vision and language but also to explicitly estimate and compensate for the current state of their own hardware. Feasible approaches include: introducing proprioceptive signals such as joint feedback, torque readings, and control frequency as additional input channels to the policy network; performing domain randomization over control parameters and calibration offsets during training so that the model learns representations insensitive to hardware changes; and introducing online system identification during deployment to estimate parameters such as gain and latency in real time and adjust action outputs accordingly. Such methods have foundations in classical robot control (e.g., adaptive control, robust MPC), but how to effectively integrate them with large-scale VLA pretraining remains an open problem.

One-line Summary

LIBERO-RIG benchmark reveals VLA model success rates plummet from 0.932 to 0.509 under hardware perturbations

Paragraph Summary

LIBERO-RIG is a benchmark specifically designed to evaluate VLA model robustness to robot hardware state changes, with its core design principle being to perturb only the robot hardware and setup while keeping LIBERO tasks completely unchanged. The benchmark covers 11 perturbation axes including camera position and optical parameters, calibration offsets, drive gain, control frequency, proprioceptive noise, base pose, table height, and gripper pads, each with 5 severity levels, combining into 210 test conditions across four LIBERO suites, evaluating nine recent VLA models at 500 episodes per condition. Results show the aggregate success rate drops from a baseline of 0.932 to 0.767 at moderate severity and 0.509 at highest severity. Degradation varies significantly across perturbation axes and action interfaces, and visual perturbation training cannot mitigate hardware failures that do not alter camera images, posing a direct challenge to current data augmentation practices.