GeoState

The paper proposes GeoState, a viewpoint-decoupled geometric representation method for predictive 3D world modeling, whose core idea is to separate scene state evolution from camera-dependent observations.

GeoState contains three key components: a scene tokenizer, a geometry decoder and an autoregressive world model; the scene tokenizer aggregates multi-view GFM features into a viewpoint-decoupled scene state; the geometry decoder queries the scene state with camera rays to render observations from a specific viewpoint; the autoregressive world model performs prediction in the decoupled scene state space and models camera motion independently.

GeoState outperforms previous 3D world models on the Waymo dataset.

GeoState supports geometrically consistent cross-view queries, that is, the same scene state can be decoded from different camera angles with geometrically matching results.

GeoState achieves zero-shot generalization on the KITTI and nuScenes datasets, transferring to different sensor configurations and scene distributions without retraining on new datasets.

The project page of the GeoState paper has been made public at https://hongchang-liu.github.io/GeoState-project-page/.

The decoupling idea of GeoState may influence broader directions such as embodied intelligence (separate modeling of agent motion and environment dynamics) and video generation (decoupling of viewpoint control and content generation).