Thinker-WM: A Unified World Model with Shared Multimodal Representations

World models are becoming a central direction in embodied intelligence research, but most existing systems are fragmented across modalities and functions: visual dynamics, geometric structure, and executable actions are typically connected through separate interfaces or downstream modules, rather than being jointly represented and evolved within a single dynamics model. Thinker-WM aims to change this landscape—it uses a shared Unified Dynamics Transformer to unify RGB, metric depth, and robot actions into the same dynamic representation, while preserving the output structure of each modality.

From Fragmentation to Unification: The Core Design of Thinker-WM

The key concept behind Thinker-WM is "shared representation." Traditional world models typically only model visual prediction, with actions, depth, and other information serving as conditional inputs or post-processing results, making it difficult for the model to truly understand the physical coupling relationships among the three. Thinker-WM's approach is: equip each modality with dedicated encoders and decoders, but have appearance, geometry, and action be converted into a unified dynamic representation at intermediate layers, jointly evolved through shared parameters.

This means the model is not simply stitching together multiple tasks, but establishing cross-modal dynamic consistency in latent space. The paper emphasizes that the same set of parameterization can support multiple functions, including joint video-action generation, world prediction, policy reasoning, action-conditioned generation, video-to-action generation, and inverse dynamics. In other words, a single model simultaneously plays the role of world simulator and policy learner, without needing to train separate modules for each capability.

Does the Shared Representation Truly "Understand" Actions?

To verify whether the shared representation truly encodes action-related information, the authors designed a lightweight motion probe. Experiments show that even with the visual representation frozen, action information can still be recovered from it. This point is crucial: it demonstrates that the model is not merely aligning action labels at the output end, but preserving motion-related structure at the internal representation level.

This recoverability provides evidence that "the shared representation has physical meaning." Compared to simply reporting generation metrics, the probe experiment more directly addresses a fundamental question—whether unified modeling is merely an engineering compromise, or whether it genuinely brings gains at the representation level.

World Model (World Model) concept originates from reinforcement learning and cognitive science, referring to an agent's internal simulator of environmental dynamics: given the current state and action, predict the next state. In embodied intelligence scenarios, world models typically need to simultaneously handle three types of information: visual observations, 3D geometry, and executable actions. Traditional approaches treat them as independent pipelines—for example, using video diffusion models for visual prediction, separate policy networks to output actions, and then coordinating through post-processing modules. The problem with this fragmentation is: the visual dynamics learned by the model may not be consistent with the physical consequences of actions, and geometric structure is often used as conditional input rather than an object of joint modeling. The "unified dynamic representation" proposed by Thinker-WM precisely targets this pain point, attempting to make appearance, structure, and behavior follow consistent dynamic laws within the same latent space.

Motion probe is a representation diagnostic technique, borrowed from the linear probe method in the self-supervised learning field. The basic idea is: freeze the model's backbone representation, train only a lightweight classifier or regressor on top of it, and attempt to recover target information (here, robot actions) from the frozen features. If the probe can recover actions with low error, it indicates that action-related information is already implicit in the representation, rather than existing only as label alignment at the output end. Compared to directly reporting generation metrics, probe experiments eliminate the possibility of "cheating" by downstream decoders, and are therefore often used as causal evidence of representation quality. In embodied world models, this technique helps distinguish between "truly learning physical coupling" and "merely fitting action labels at the output layer."

Training Data and Experimental Performance

Thinker-WM's training corpus is considerable in scale, totaling approximately 20K hours of first-person video, 10K hours of real robot data, and 1K hours of simulation data, of which approximately 4K hours of real robot data comes with aligned depth supervision. This heterogeneous mixture is uncommon in the embodied domain: first-person video provides rich behavioral priors, real robot data ensures action usability, simulation data supplements controllability, and depth supervision truly anchors geometric structure to physical space.

In embodied world model evaluations, Thinker-WM ranks first on EWMBench, WorldOlympiad, and WorldArena. These benchmarks examine the prediction quality and action consistency of world models from different dimensions, and leading across multiple benchmarks is more convincing than a single metric.

Gains Revealed by Controlled Experiments

Beyond benchmark results, the paper also conducted controlled studies examining the effects of action conditioning, geometric interaction, and multi-function training. Results show that all three positively contribute to the shared representation: action conditioning makes predictions more consistent with robot behavior distributions, geometric interaction improves spatial consistency, and multi-function joint training strengthens the generality of the representation.

These ablation results are more informative than simple rankings. They indicate that Thinker-WM's advantages do not come from a single trick, but from the systematic benefits brought by the unified representation paradigm itself.

Metric depth refers to depth values expressed in real physical units (such as meters), rather than relative depth or disparity. In embodied intelligence, metric depth is crucial: robot grasping, obstacle avoidance, and navigation all require knowing the absolute distance between objects and the camera. 4K hours of real robot data with aligned depth supervision means the model can obtain pixel-level 3D geometric constraints during training, not just 2D image prediction. This supervision anchors visual representations to physical space, making the geometric structure learned by the model consistent with real-world scales. Compared to using only relative depth or monocular estimated depth, metric depth can significantly improve the accuracy of spatial reasoning and action planning.

Ablation study is a standard method in machine learning for attributing model performance sources: by removing or replacing a component and observing performance changes, one can judge that component's contribution. In Thinker-WM, the authors separately control three variables: action conditioning, geometric interaction, and multi-function training, and results show that all three positively contribute to the shared representation. This controlled study is more informative than simple benchmark rankings, because it can reveal whether performance improvements come from the unified representation paradigm itself or from a single trick. The design of ablation experiments needs to ensure other conditions are consistent, otherwise conclusions may be contaminated by confounding variables, so papers typically need to detail the specific settings of controlled variables.

Implications for Embodied Intelligence

The value of Thinker-WM is not just a set of benchmark results. It points to a direction: world models for embodied intelligence do not need to split vision, geometry, and action into independent modules; shared dynamic representations may be a more natural choice. When the model evolves appearance, structure, and behavior in the same latent space, policy reasoning and world prediction are no longer two separate systems, but different readouts of the same capability.

Of course, unification also brings challenges—the data distributions, temporal scales, and supervision densities of different modalities differ greatly, and how to maintain the output quality of each modality under shared parameters remains a question for future work. Thinker-WM's answer is: use modality-specific encoders and decoders to preserve structure, and use a shared Transformer to carry dynamics. This path is worth continued validation.

One-line Summary

Thinker-WM uses a shared Unified Dynamics Transformer to unify RGB, depth, and action, ranking first on multiple embodied world model benchmarks.

Paragraph Summary

Thinker-WM addresses the fragmentation of vision, geometry, and action in embodied intelligence world models by proposing a shared Unified Dynamics Transformer that unifies RGB, metric depth, and robot actions at the representation level, while preserving modality-specific encoders and decoders. The model supports six functions—joint video-action generation, world prediction, policy reasoning, action-conditioned generation, video-to-action generation, and inverse dynamics—sharing the same set of parameters. Motion probe experiments show that action information can still be recovered from frozen visual representations, proving that the shared representation encodes motion structure internally. The training corpus includes approximately 20K hours of first-person video, 10K hours of real robot data, and 1K hours of simulation data, with approximately 4K hours featuring aligned depth supervision. It ranks first on EWMBench, WorldOlympiad, and WorldArena, and ablation experiments show that action conditioning, geometric interaction, and multi-function training all bring gains.