SPW-Nav
A language-driven streaming panoramic world model that interprets natural language movement instructions and generates 360-degree panoramic video in a streaming manner in real time. It supports immediate instruction switching and long-horizon continuous navigation.
Timeline (last 90 days)
SPW-Nav is a world model capable of understanding movement instructions and generating one-minute 2K-level 360-degree videos in real time in a streaming manner from a single panoramic image.
SPW-Nav is proposed in a paper on arXiv titled "SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation."
SPW-Nav interprets each language instruction as a corresponding camera displacement and rotation within the panoramic image generated from the previous frame, thereby driving continuous changes in the scene.
SPW-Nav proposes three technical designs: spherical rotation decoupling, pose-aligned conditioning, and a multi-memory few-step generator.
Spherical rotation decoupling directly models the rotation transformation as a rigid body motion on the sphere (an operation on the SO(3) group) and then projects it back to the plane, rather than directly manipulating planar pixels.
Pose-aligned conditioning constrains translation inputs within a bounded range, thereby maintaining stability during long-sequence generation.
Multiple memory mechanisms combined with a few-step generator enable smooth scene continuation rather than abrupt switching when instructions change, supporting instant instruction switching capability.
The research team constructed a dedicated dataset, SPW-NavSet, containing panoramic videos with camera trajectories and verified instructions.
Under language-driven conditions, SPW-Nav outperforms previous panoramic generators on both camera-following accuracy and video quality metrics.
As a newly released preprint, SPW-Nav's real-world generalization capability, stability in complex scenes, and computational cost still await further validation by the community.
All Facts (10)
SPW-Nav is a world model capable of understanding movement instructions and generating one-minute 2K-level 360-degree videos in real time in a streaming manner from a single panoramic image.
50%UnverifiedSPW-Nav is proposed in a paper on arXiv titled "SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation."
50%UnverifiedSPW-Nav interprets each language instruction as a corresponding camera displacement and rotation within the panoramic image generated from the previous frame, thereby driving continuous changes in the scene.
50%UnverifiedSPW-Nav proposes three technical designs: spherical rotation decoupling, pose-aligned conditioning, and a multi-memory few-step generator.
50%UnverifiedSpherical rotation decoupling directly models the rotation transformation as a rigid body motion on the sphere (an operation on the SO(3) group) and then projects it back to the plane, rather than directly manipulating planar pixels.
50%UnverifiedPose-aligned conditioning constrains translation inputs within a bounded range, thereby maintaining stability during long-sequence generation.
50%UnverifiedMultiple memory mechanisms combined with a few-step generator enable smooth scene continuation rather than abrupt switching when instructions change, supporting instant instruction switching capability.
50%UnverifiedThe research team constructed a dedicated dataset, SPW-NavSet, containing panoramic videos with camera trajectories and verified instructions.
50%UnverifiedUnder language-driven conditions, SPW-Nav outperforms previous panoramic generators on both camera-following accuracy and video quality metrics.
50%UnverifiedAs a newly released preprint, SPW-Nav's real-world generalization capability, stability in complex scenes, and computational cost still await further validation by the community.
50%