VLA

VLA directly connects pre-trained vision-language models with robot action policies, enabling robots to process natural language instructions, camera images, and low-level motion control in a unified manner.

VLA models integrate visual perception, language understanding, and robot action planning into a unified model, typically using a pretrained large vision-language model as the backbone, with representative works including Google's RT-2 and OpenVLA.

The current mainstream technical approach to embodied intelligence is the Vision-Language-Action model (VLA), which takes camera images and language instructions as joint inputs and directly outputs joint control signals for robotic arms.

RT-2 first demonstrated the feasibility of the VLA architecture by discretizing robot actions into tokens and having the vision-language model directly output action sequences.

DeepMind developed the VLA (Vision-Language-Action model) system, which treats the robot's physical actions on equal footing with visual tokens and language tokens.

Google DeepMind的RT-2首次将视觉-语言模型直接用于生成机器人动作指令

VLA (Vision-Language-Action) models are a class of multimodal models that unify visual perception, natural language understanding, and action generation in a single end-to-end architecture, typically taking image sequences and language instructions as input and directly outputting robot joint angles, end-effector poses, or discrete action tokens.

In the context of AdaRSI, the executor refers to the neural network body that carries the action generation capability (usually the weights of a VLA model), while the tool harness is the scheduling and orchestration layer wrapped around the executor.

Typical representatives of VLA models include RT-2, OpenVLA and GR00T, which are usually pretrained jointly on internet-scale image-text data and large-scale robotic manipulation data.

Pretrained vision-language-action models (VLA) tend to lose coherence midway and fail when faced with unseen long-horizon tasks.

Across three VLA models, approach-aware episode conditioning consistently exceeds naive behavior cloning in average success rate.

Typical representatives of action models are vision-language-action models (VLA), such as RT-2 and OpenVLA, which usually use a pretrained multimodal large model as the backbone and, after fine-tuning on robot data, directly output joint positions, end-effector trajectories or discrete action tokens.

Directions such as LLM, Agentic, VLA, multimodal, robotics, or foundation models have become the absolute focus of industry ML research since 2023, with many related internship positions and high salaries, but fierce competition.

A generalist policy refers to a robot policy model pretrained on multiple tasks and scenarios that can be transferred to new instructions, usually based on a vision-language-action (VLA) architecture; existing generalist policies often can only achieve per-task fine-tuning after pretraining, and true zero-shot generalization remains an open problem.

VioLA's method reproduces the same effect on two VLA backbones and one world-action model backbone, indicating that its approach does not depend on a particular model architecture.

The plug-in module, without changing the parameter structure of the pretrained VLA backbone, additionally inserts lightweight encoders or adapters and injects scene tokens as additional conditions into the feature stream of the policy network; its training cost is low and transfer is fast, but its expressive capacity is limited by the adapter capacity.

CARE's certified accelerator selection delivers up to 10× speedup for VLA deployment while ensuring bounded failure risk.

RealtimeWAM can simultaneously predict future visual dynamics and robot actions in a single forward pass, and achieves near-lossless one-step inference through asynchronous teacher-anchored distillation.

Representative VLA architectures include Google's RT-2 and Physical Intelligence's π0; the core challenge of VLA lies in the fact that VLM autoregressive decoding speed is lower than the 10–50 Hz real-time frequency required by the robot control loop.

VLA models often lack intrinsic modeling of the dynamic evolution of the physical world, and are more often imitating action mappings rather than predicting world changes.