CondenseVLA: Learnable History Compression Makes Multi-Frame VLAs More Efficient

Vision-Language-Action models (VLAs) are becoming mainstream in robotic manipulation, but single-frame observations often cannot resolve ambiguous cues such as brief contacts or temporary occlusions, making temporal information integration essential. Expanding the history window is an intuitive solution, but it brings surging computational costs and redundant sampling issues. CondenseVLA proposes a learnable decoder-side history compression module that attempts to efficiently leverage multi-frame visual context under a limited computational budget.

The Bottleneck of Multi-Frame VLAs: Is a Larger History Window Always Better?

The short-term cues that robotic manipulation relies on—such as the contact at the moment of grasping, the reappearance of an object after occlusion, and changes in intermediate states—are often ambiguous in single-frame images. Intuitively, expanding the history window provides more context, but the original material points out that this approach is computationally expensive and often yields limited gains due to observation redundancy and fixed, task-agnostic sampling strategies.

In other words, the question is not "how many frames to look at" but "which frames to look at and how to integrate them." CondenseVLA's starting point is precisely to abandon brute-force window expansion and instead learn to compress history.

VLA (Vision-Language-Action models) are a class of policy models that jointly model visual perception, natural language instructions, and robot action generation. A typical architecture uses a pretrained vision-language model (VLM) as its backbone, encoding camera images and task text into multimodal features, which are then used by an action decoding head to output end-effector poses or joint control values for the robotic arm. The core advantage of VLAs lies in leveraging semantic priors from internet-scale pretraining, enabling robots to generalize to unseen objects and instructions. However, standard VLAs typically take a single frame or a small number of frames as input and lack explicit modeling of temporal continuity, making them prone to ambiguity in scenarios that depend on short-term dynamics such as grasping contact and occlusion recovery. It is against this backdrop that CondenseVLA explores how to inject multi-frame history information into VLAs without significantly increasing computational burden.

CondenseVLA's Core Design: Learnable Compression on the Decoder Side

CondenseVLA is a multi-frame VLA policy whose key component is a decoder-side history compression module. The specific pipeline is as follows: the VLM backbone first caches the feature sequence of each frame, and then CondenseVLA uses a set of learnable queries and multi-head cross-attention to distill the observation history into a fixed number of slot tokens.

The ingenuity of this design lies in the fact that compression occurs at the decoding stage rather than the encoding stage—the VLM backbone does not need to recompute for each frame, and history information enters action decoding in the form of lightweight tokens. The fixed number of slots also means that computational overhead does not grow linearly with the history window, leaving room for real-time control.

Auxiliary Regularization: Making Slot Roles Clear

If the model is left to learn freely, multiple slots tend to focus on similar regions, causing information duplication. CondenseVLA introduces simple auxiliary regularization: on one hand, it penalizes overlap between slot attention maps; on the other hand, it decorrelates slot embeddings, encouraging each slot to focus on complementary visual evidence.

This regularization adds no inference burden but significantly improves compression efficiency. The original material describes it as a "simple" auxiliary loss, indicating that the method does not rely on complex structures but instead guides slot specialization through training signals.

Learnable queries and cross-attention are common mechanisms in Transformers for information aggregation. A set of randomly initialized query vectors, updated during training, adaptively extract information from an input sequence (here, the feature sequence of history frames) through multi-head cross-attention, outputting a fixed number of tokens. Unlike pooling or fixed sampling, query vectors can learn "which frames and which regions to focus on" based on task loss signals, thereby achieving content-aware compression. In CondenseVLA, the number of slot tokens is a preset hyperparameter that does not vary with the length of the history window, so the decoder's computational cost remains constant. This design shifts the decision of "which history frames to select" from manual rules to model training, which is the key to the method's efficient multi-frame inference.

Attention map overlap penalty and embedding decorrelation are two common regularization techniques for representation diversity. The attention map overlap penalty is typically implemented by minimizing the cosine similarity or dot product between different slot attention distributions, forcing each slot to focus on different spatial regions or temporal segments in the history frames. Embedding decorrelation applies a penalty to the covariance matrix of slot vectors, reducing linear correlations between slot representations and preventing information redundancy. Neither changes the network structure at inference time—they are only added as auxiliary losses during training, so they do not increase deployment overhead. Such regularization has precedents in visual question answering, object detection, and video understanding. CondenseVLA transfers it to the history compression scenario, achieving specialized slot division of labor at a small engineering cost.

Experimental Results: Consistent Improvements Across Multiple Tasks

CondenseVLA provides quantitative results on multiple evaluation benchmarks:

• Simpler-WidowX: success rate improved from 58.7% to 69.8%, an absolute gain of 11.1 percentage points;
• LIBERO: reached 97.2%;
• Real-world tasks: reached 52.8%;
• Memory-dependent RMBench: success rate of 21.6%.

These numbers span both simulation and real-world scenarios, indicating that selectively compressing recent visual evidence does indeed help action decoding. However, the 21.6% success rate on RMBench also suggests that tasks requiring long-term memory or multi-observation reasoning remain a weakness. The original material explicitly acknowledges that harder multi-observation tasks expose the limitations of a finite history window.

The Ceiling of Finite Windows and Future Directions

CondenseVLA's value lies in validating a direction: rather than infinitely expanding the history window, it is better to let the model learn to selectively compress recent evidence. The fixed-slot design strikes a balance between efficiency and performance, but the 21.6% RMBench result shows that when tasks require spanning longer time horizons or more complex observation combinations, the strategy of finite windows plus compression will still hit a ceiling.

The project page is now public (https://github.com/FeiyangHong/CondenseVLA). If hierarchical or adaptive window mechanisms can be introduced into the compression module in the future, it may further break through the bottleneck of memory-dependent tasks. For researchers and engineers concerned with VLA efficiency and multi-frame inference, CondenseVLA offers a lightweight solution worth referencing.