GeoState: View-Decoupled Geometric State Learning for 3D World Modeling

In recent years, 3D World Models have increasingly leveraged features extracted by Geometry Foundation Models (GFM) as predictive world states. Such features carry rich geometric priors, and in theory should enable models to better understand the structure and dynamics of three-dimensional scenes. However, GFM features themselves suffer from a long-overlooked problem: view dependency. When the same scene is observed from different camera angles, the extracted features change—meaning the scene's own dynamics and the changes induced by camera motion become entangled, making the evolution of the world state difficult to predict.

To address this contradiction, a new paper proposes GeoState, a view-decoupled geometric representation method for predictive 3D world modeling. Its core idea is to separate the evolution of scene state from camera-dependent observations, so that the world model is no longer disturbed by viewpoint changes when predicting the future.

Root of the Problem: View Dependency Makes World States "Impure"

To understand the value of GeoState, one must first recognize the dilemma of existing methods. Current mainstream 3D world models typically treat GFM features directly as world states—the model extracts geometric features at each frame and then predicts the next frame's state based on these features. The hidden risk of this approach is that GFM features do not solely reflect the scene itself; they are also overlaid with the influence of camera parameters.

A straightforward example: a car waits at a red light at an intersection, and the scene itself undergoes no motion. But if the camera is moving, the GFM features will keep changing. If the model treats these changes as "the world state evolving," it will learn incorrect dynamics. More seriously, when the model needs to predict multiple future frames, each frame has a different camera pose, and the feature scale will also drift, forcing the model to repeatedly perform scale alignment during rollout, further increasing prediction difficulty.

The paper points out that this entanglement is precisely the key bottleneck that makes state evolution difficult to predict.

Geometry Foundation Models are a class of models pre-trained on large-scale 3D data or multi-view data that can output geometry-aware feature representations. Representative works include pre-trained backbones based on multi-view stereo, point clouds, or neural radiance fields. Their advantage is that the features implicitly encode 3D priors such as depth, surface normals, and spatial correspondence, so they are often used directly as inputs to downstream world models or perception systems. But such priors are learned under a specific camera projection—that is, each feature output by the network is tightly bound to the imaging geometry of the current viewpoint. When the camera position or intrinsics change, even if the scene is static, the position and occlusion relationships of the same spatial point projected onto the image will change, causing systematic shifts in the features. This is the model-level source of what the paper calls "view dependency."

Autoregressive World Models refer to generative models that take the current or historical state as input and progressively predict future states. A common approach treats the scene representation at each moment as a token sequence, using structures such as Transformers to reason in temporal order. The key challenge is that when the state representation itself varies with observation conditions, the model will mistakenly attribute observational changes to scene evolution and will accumulate errors over multi-step rollouts. In addition, feature distribution drift between frames forces the model or training pipeline to introduce extra alignment operations, such as per-frame normalization or adversarial scale constraints, which both increase complexity and weaken generalization. Understanding these two points helps clarify why GeoState redefines the "state" as a view-independent variable.

GeoState's Core Design: Scene Tokenizer and Geometry Decoder

GeoState's approach can be summarized in one sentence: separate "what the scene is" from "how the camera sees it" into two independent parts.

Specifically, the system contains several key components:

The Scene Tokenizer is responsible for aggregating multi-view GFM features into a view-decoupled scene state. This state only describes the scene itself and does not carry observation-angle information specific to any camera.

The Geometry Decoder works in the opposite direction—it uses camera rays to query the scene state, thereby rendering observations from a specific viewpoint. In other words, the camera is no longer a component of the state, but merely a query condition during decoding.

The Autoregressive World Model performs prediction in the decoupled scene state space, independently modeling camera motion. Since the scene state itself is view-independent, the model can decode future states under any camera specified during prediction or externally, while maintaining a single context scale throughout the rollout without per-frame scale alignment.

A direct benefit of this architecture is that the model learns "cleaner" scene dynamics, and prediction difficulty is significantly reduced.

The "tokenizer" in the Scene Tokenizer borrows the concept from natural language processing and vision Transformers, originally referring to a module that splits raw input and maps it into discrete or continuous token sequences. In GeoState, it does not discretize; instead, it compresses GFM features from multiple camera views through attention or aggregation operations into a set of compact, view-independent scene tokens. These tokens can be understood as a "canonical representation" of the scene: no matter which camera observes it, the same scene should map to the same or similar state. The aggregation process typically needs to use camera intrinsics and extrinsics to back-project features from each view into a unified 3D reference frame and then remove camera-related components, so the tokenizer's design directly determines how thoroughly view decoupling is achieved.

The Geometry Decoder, on the other hand, takes on the opposite responsibility from the tokenizer: given a set of view-independent scene tokens and arbitrary target camera parameters, it generates geometric observations from that camera viewpoint, such as depth maps, feature maps, or RGB images. Its core mechanism is camera rays, i.e., direction vectors starting from the camera's optical center and passing through each pixel. The decoder uses these rays as queries, sampling information at corresponding positions from the scene state via ray casting or cross-attention. Since the camera only appears in the decoding stage and the state itself contains no camera information, the same state can be used to render multiple different viewpoints while naturally preserving geometric consistency. This asymmetric structure of "encoding aggregation, decoding query" is the key engineering approach by which GeoState achieves decoupling.

Experimental Results: Performance Advantage on Waymo and Cross-Dataset Generalization

The paper validates GeoState's effectiveness on multiple autonomous driving datasets.

On the Waymo dataset, GeoState outperforms previous 3D world models. Given the complexity of Waymo scenes and the multi-camera configuration, this result indicates that view decoupling indeed brings improvements in prediction accuracy.

More noteworthy is the geometric consistency of cross-view queries. Because the scene state is decoupled from the viewpoint, GeoState supports geometrically consistent cross-view queries—in other words, the same scene state can be decoded from different camera angles, and the decoded results align geometrically with each other. This is difficult for traditional view-entangled methods to achieve.

Zero-shot generalization is another highlight. GeoState achieves zero-shot generalization on KITTI and nuScenes, meaning the model can transfer to different sensor configurations and scene distributions without retraining on the new datasets. For real-world deployment, this kind of generalization is more valuable than performance gains on a single dataset.

Zero-shot Generalization refers to a model's ability to transfer directly to a new dataset with usable performance without any training or fine-tuning on the target dataset. For autonomous driving 3D world models, this means the model does not depend on a specific sensor configuration, number of cameras, or scene distribution, and can work on datasets with significant differences such as KITTI and nuScenes. Zero-shot capability usually comes from the decoupling and normalization of the representation itself: when the scene state is not bound to specific camera parameters, switching datasets only changes the camera query conditions during decoding without destroying the learned dynamics. Therefore, GeoState's zero-shot results are not only performance metrics but also indirectly validate the generality of the view-decoupled representation. It should be noted, however, that zero-shot transfer may still be limited by differences between the two datasets in scene types, annotation density, and motion patterns, and practical deployment often still requires a small amount of adaptation.

Why This Approach Is Worth Attention

The value of GeoState lies not only in metric improvements, but also in the fact that it redefines what "world state" should contain.

Past approaches modeled cameras and scenes together, essentially using a coupled representation to approximate two independent physical processes. GeoState's approach aligns with physical intuition: the evolution of the scene follows its own laws, and camera motion is the observer's choice. Only by separating the two can a model learn truly transferable and predictable world dynamics.

This decoupling idea may also influence broader directions. For example, in embodied intelligence, the agent's motion and the environment's dynamics likewise need to be modeled separately; in video generation, decoupling viewpoint control from content generation is also an active topic. GeoState provides an architectural paradigm worth referencing: use a tokenizer to aggregate scenes, a decoder to query viewpoints, and an autoregressive model to predict states.

The project page has been made public: https://hongchang-liu.github.io/GeoState-project-page/ . Researchers interested in this direction can further examine implementation details and visualization results.

One-line Summary

GeoState decouples scene evolution from camera observation, enabling 3D world models to predict the future more accurately under view-independent states.

Paragraph Summary

GeoState is a paper addressing the view dependency of geometry foundation model features in 3D world models. Existing methods treat GFM features directly as world states, causing scene dynamics and camera motion to become entangled, making multi-step rollouts difficult to predict and requiring repeated scale alignment. GeoState proposes separating "what the scene is" from "how the camera sees it": a scene tokenizer aggregates multi-view features into a view-decoupled state, a geometry decoder uses camera rays to query the state and render specific viewpoints, and an autoregressive world model independently models camera motion in the decoupled state space. On Waymo, GeoState surpasses existing 3D world models, supports geometrically consistent cross-view queries, and generalizes zero-shot to KITTI and nuScenes. Its core value lies in redefining world state as containing only the scene itself, offering a reference architecture for decoupled modeling in embodied intelligence and video generation.