PPMem
A 3D memory method for video world models that uses planar primitives as a persistent memory representation. A diffusion transformer jointly predicts planar geometry, yielding compact, explicitly geometric, cross-view-consistent and adaptive scene memory.
Timeline (last 90 days)
PPMem uses a set of oriented plane primitives as persistent 3D memory, with each primitive requiring only 7 parameters.
PPMem can represent a complete scene with about 100,000 primitives in only about 5 MB, three orders of magnitude more compact than per-pixel point clouds.
PPMem adds a plane prediction head and a Depth Token Fusion (DTF) module to a diffusion backbone, enabling each autoregressive video block to output plane geometry parameters while generating RGB frames.
PPMem merges and refines the planes predicted by the new block with historical planes, then feeds them back to the next block in the form of rendered depth, achieving rolling updates of persistent memory.
PPMem's plane memory has four key properties: compactness, explicit geometry, cross-view consistency, and adaptivity.
On the DL3DV and RealEstate10K datasets, PPMem outperforms previous 3D-grounded world models in video quality while jointly outputting metric-scale depth, surface normals, and exportable plane geometry.
PPMem can maintain coherent consistency over hundreds of frames of generation under large viewpoint changes.
The PPMem project homepage has been made public, where the code and more visualization results can be further consulted.
All Facts (8)
PPMem uses a set of oriented plane primitives as persistent 3D memory, with each primitive requiring only 7 parameters.
50%UnverifiedPPMem can represent a complete scene with about 100,000 primitives in only about 5 MB, three orders of magnitude more compact than per-pixel point clouds.
50%UnverifiedPPMem adds a plane prediction head and a Depth Token Fusion (DTF) module to a diffusion backbone, enabling each autoregressive video block to output plane geometry parameters while generating RGB frames.
50%UnverifiedPPMem merges and refines the planes predicted by the new block with historical planes, then feeds them back to the next block in the form of rendered depth, achieving rolling updates of persistent memory.
50%UnverifiedPPMem's plane memory has four key properties: compactness, explicit geometry, cross-view consistency, and adaptivity.
50%UnverifiedOn the DL3DV and RealEstate10K datasets, PPMem outperforms previous 3D-grounded world models in video quality while jointly outputting metric-scale depth, surface normals, and exportable plane geometry.
50%UnverifiedPPMem can maintain coherent consistency over hundreds of frames of generation under large viewpoint changes.
50%UnverifiedThe PPMem project homepage has been made public, where the code and more visualization results can be further consulted.
50%