How Do Latent World Models Learn Intrinsic Dynamics? New Research Reveals the Key Role of Action Predictability

Core Problem: When Visual Variation Masks Control Signals

The core task of a World Model is to understand the relationship between actions and their consequences—that is, intrinsic dynamics. However, in real-world scenarios, changes in visual appearance (such as lighting, texture, background, and other distractors unrelated to control) can severely interfere with the learned representations, making it difficult for the model to capture the essential connection between actions and outcomes.

The real-world impact of this problem is significant. Imagine a robot performing a pushing task on tables of different colors. If the encoder encodes the table color differences into the latent state, then a planner trained on these states will mistake color variation for part of the dynamics, leading to planning failures.

World Models are a class of neural network models that learn the dynamic patterns of an environment and can simulate future states in latent space. They typically consist of an encoder, a dynamics model, and a planner. The typical training pipeline is: the encoder compresses high-dimensional observations (e.g., images) into low-dimensional latent states; the dynamics model predicts the next latent state given an action; and the planner searches for action sequences in latent space to achieve a goal. Since planning relies entirely on state distances in latent space, which information the encoder retains and which it discards directly determines planning quality. In robot learning, observations often contain a large amount of visual factors unrelated to control, such as lighting, texture, and background color. These factors are called distractors. If the encoder encodes these distractors into the latent state, the dynamics model must additionally model their patterns of change, thereby consuming representation capacity that should be devoted to learning action-consequence relationships. This ultimately causes the planner to fail when the visual distribution shifts.

Theoretical Insight: The Constraint Relationship Between Action Predictability and Intrinsic Dynamics

This work proposes a core hypothesis: encouraging action predictability can prevent the representation from discarding the distinguishing information between different actions and their consequences. Action predictability refers to the ability to reconstruct actions from consecutive latent states.

The researchers theoretically derive a key conclusion: when the encoder merges distinguishable outcomes of different actions under the same state, as well as different appearances, the squared inverse prediction error has a positive lower bound. In other words, if the encoder confuses different outcomes caused by different actions, then errors are inevitable when inferring actions from latent states. This theoretical result reveals an important constraint: forward prediction can remain accurate on these confused transitions, which means that the forward prediction loss alone is insufficient to preserve control-relevant distinguishing information, while action prediction provides an additional inductive bias.

Inverse Dynamics Models (IDMs) are a class of models that infer the executed action from observations or latent states at two adjacent time steps—that is, they learn the mapping from state transitions back to actions. In contrast, Forward Dynamics Models predict the next state based on the current state and action. Inverse dynamics models have unique value in representation learning: to accurately infer actions, the model must retain information that can distinguish the consequences of different actions, and this information is precisely what is highly relevant to intrinsic dynamics. Action predictability uses this capability as a training objective. By using inverse dynamics prediction error as an auxiliary loss, it forces the encoder to retain action-distinguishable state information. This idea is related to the information bottleneck principle in contrastive learning: when a model needs to complete a prediction task, the encoder is forced to retain the minimal sufficient information necessary to complete the task, thereby naturally filtering out task-irrelevant distractors.

Experimental Validation: Multi-Dimensional Evidence Supports the Core Hypothesis

The research team validated the relationship between action predictability and the preservation of intrinsic dynamics through three complementary experimental approaches:

Controlled Appearance Diversity Comparison

By systematically controlling the degree of appearance variation, the researchers observed a stable positive correlation between action prediction capability and the preservation of intrinsic dynamics. The more diverse the appearances, the more critical the constraint of action prediction on representation learning.

Supervised Intervention Experiments

By intervening in the supervisory signal of action labels (increasing, decreasing, or removing them), the study confirmed the causal role of action labels in maintaining control-relevant transition information.

Latent Probe Analysis

Using linear probes to examine control-relevant information encoded in latent states, the results showed that action predictability training indeed enabled the representation to retain more control-relevant distinguishing information, rather than being dominated by visual variation.

Concat IDM: A Simple and Effective Implementation

The researchers proposed Concat IDM (Concatenated Inverse Dynamics Model) as a concrete method for achieving action predictability. Its core design idea is:

• Concatenate the latent states of two consecutive time steps and feed them into the inverse dynamics model.
• Use the existing action labels in offline trajectories to update the shared encoder.
• Do not change the planner structure at test time, ensuring the method is plug-and-play.

The ingenuity of this design lies in the fact that it requires no additional labeled data (offline trajectories naturally contain action labels) and no modification to the planning algorithm. It improves representation quality simply by adding an auxiliary training objective. The concatenation operation provides the inverse dynamics model with complete information from two time steps, enabling it to more accurately distinguish state changes caused by different actions.

Offline Trajectories refer to pre-collected and stored sequences of interaction data. Each trajectory typically consists of a series of observation-action-next observation tuples. In robot learning, collecting offline datasets is far less costly than online interaction, so how to fully utilize offline data becomes a key issue. The core advantage of Concat IDM is that it directly reuses the action labels naturally carried by offline trajectories, without requiring additional human annotation or environment interaction. This idea of using implicit supervisory signals in existing data for auxiliary training has been widely applied in self-supervised learning. Auxiliary tasks are additional training objectives that share the encoder with the main task but have independent prediction heads. Their gradients are backpropagated to the shared encoder, thereby shaping a better representation. The inverse dynamics prediction head in Concat IDM is such an auxiliary task. Its gradient signal works in concert with the main task (forward dynamics prediction) to jointly optimize the encoder, enabling the latent representation to be both forward-predictable and action-distinguishable.

Experimental Results: Substantial Improvement in Planning Success Rate

On the Push-T task, the researchers conducted three independent training runs using 24 different appearances, with striking results:

• Planning success rate improved from 9.1% to 77.2%.
• Compared with the improvement under a single appearance condition, the paired improvement was 52.9 percentage points higher.

These results convey two important messages. First, Concat IDM brings a huge performance leap in visually diverse scenarios, indicating that action predictability is indeed an effective means of improving latent planning robustness. Second, the improvement under multi-appearance conditions is much greater than under single-appearance conditions, indicating that the value of this method is fully realized precisely when visual variation is rich. When there are fewer visual distractions, the model itself can already learn a reasonably good representation, and the marginal benefit of the additional constraint from action prediction is relatively limited.

Significance and Outlook

The contribution of this work can be understood on two levels. At the theoretical level, it establishes a mathematical connection between action predictability and the preservation of intrinsic dynamics, providing a new analytical framework for understanding representation learning in world models. At the practical level, Concat IDM offers a simple, general, and annotation-free method for improving planning performance under visual diversity.

For developers working on robot learning, autonomous driving, and embodied intelligence research, the value of this method lies in its extremely low implementation cost—simply adding an inverse dynamics prediction head to an existing training pipeline can significantly improve representation quality. This provides a pragmatic technical path for building more robust world models.

One-line Summary

By adding an inverse dynamics prediction head to encourage action predictability, Concat IDM improves planning success rate from 9.1% to 77.2% in visually diverse scenarios.

Paragraph Summary

This article explores the problem that world models struggle to learn intrinsic dynamics under interference from visual appearance changes. The study proposes a core hypothesis: encouraging action predictability can prevent the representation from discarding distinguishing information between different actions and their consequences. Theoretical analysis proves that when the encoder confuses the outcomes of different actions, there is a positive lower bound on inverse prediction error, while forward prediction can still remain accurate, indicating that forward prediction loss alone is insufficient to preserve control-relevant information. Based on this, the researchers propose the Concat IDM method, which concatenates latent states from two consecutive time steps and feeds them into an inverse dynamics model, using existing action labels from offline trajectories to update the shared encoder without changing the planner structure at test time. Experiments on the Push-T task using 24 different appearances show that planning success rate rises sharply from 9.1% to 77.2%, with the improvement under multi-appearance conditions exceeding that under single-appearance conditions by 52.9 percentage points, validating the effectiveness and extremely low implementation cost of action predictability as an inductive bias.