WorldRecon
A generative multi-view 3D reconstruction framework that uses a noisy global point cloud as a geometric scaffold to guide a native 3D generative model. It reconstructs large-scale scene meshes from images without calibrated poses and outputs both geometry and PBR materials.
Timeline (last 90 days)
WorldRecon is a generative multi-view reconstruction framework that can reconstruct large-scale 3D scene meshes from RGB images with uncalibrated camera poses, balancing geometric fidelity and generative completion.
The first step of WorldRecon's method is to use a reconstruction model to estimate camera poses from the input images and obtain a coarse, noisy global point cloud, then partition this point cloud into multiple overlapping spatial blocks, turning large-scene generation into a local block generation problem.
On top of the native 3D generative model TRELLIS.2, WorldRecon designs a dedicated reconstruction-style conditioning architecture, in which each spatial block generation simultaneously receives a local point cloud crop and the corresponding image observations as input.
WorldRecon's conditioning mechanism enables the model to simultaneously output sparse structure, fine geometry and PBR materials.
WorldRecon introduces neighbour-aware latent inpainting together with a causal world generation process, allowing structure, geometry and material information to be passed sequentially between overlapping blocks to maintain scene coherence.
In synthetic and real indoor scene experiments, compared with the strongest baseline, WorldRecon reduced Chamfer distance by 14.9% and 43.1% on the two test sets, respectively, while improving F-score by 17.3% and 12.0%.
WorldRecon still relies on the initial pose estimation and global point cloud as scaffolding, the quality of prior reconstruction directly affects the upper bound of the final result, and block overlap and cross-block propagation bring additional computational overhead.
The WorldRecon project homepage has been made public, containing more reconstruction results and implementation details.
All Facts (8)
WorldRecon is a generative multi-view reconstruction framework that can reconstruct large-scale 3D scene meshes from RGB images with uncalibrated camera poses, balancing geometric fidelity and generative completion.
50%UnverifiedThe first step of WorldRecon's method is to use a reconstruction model to estimate camera poses from the input images and obtain a coarse, noisy global point cloud, then partition this point cloud into multiple overlapping spatial blocks, turning large-scene generation into a local block generation problem.
50%UnverifiedOn top of the native 3D generative model TRELLIS.2, WorldRecon designs a dedicated reconstruction-style conditioning architecture, in which each spatial block generation simultaneously receives a local point cloud crop and the corresponding image observations as input.
50%UnverifiedWorldRecon's conditioning mechanism enables the model to simultaneously output sparse structure, fine geometry and PBR materials.
50%UnverifiedWorldRecon introduces neighbour-aware latent inpainting together with a causal world generation process, allowing structure, geometry and material information to be passed sequentially between overlapping blocks to maintain scene coherence.
50%UnverifiedIn synthetic and real indoor scene experiments, compared with the strongest baseline, WorldRecon reduced Chamfer distance by 14.9% and 43.1% on the two test sets, respectively, while improving F-score by 17.3% and 12.0%.
50%UnverifiedWorldRecon still relies on the initial pose estimation and global point cloud as scaffolding, the quality of prior reconstruction directly affects the upper bound of the final result, and block overlap and cross-block propagation bring additional computational overhead.
50%UnverifiedThe WorldRecon project homepage has been made public, containing more reconstruction results and implementation details.
50%