CoTL
A method for learning 3D scene representations with compact latent tokens. An encoder compresses multi-view images into a small number of tokens, and a query-based Gaussian decoder then recovers renderable 3D Gaussian primitives, supporting both reconstruction and generation.
Timeline (last 90 days)
CoTL (Compact Token Learning) proposes using a small set of compact latent Tokens to preserve the geometric and appearance information of a scene, allowing representation capacity to scale naturally with the number of observations.
CoTL's encoder takes a set of pose-free multi-view images as input, compresses each image into a small set of latent Tokens, and aggregates information across views to form a view-consistent scene representation.
In CoTL's encoder, each additional view only requires extending the representation by a small portion of resolution-independent Tokens, allowing the representation's capacity to grow flexibly with the number of available observations rather than being limited by fixed-size network parameters.
CoTL uses a query-based Gaussian decoder that, through scene-agnostic learnable query vectors, accesses the Tokens produced by the encoder via an attention mechanism to recover a set of explicitly renderable 3D Gaussian primitives; novel-view rendering supervision signals train the entire compact representation end-to-end.
The paper conducted experiments on datasets such as ACID, and the results show that CoTL's compact, unstructured Token set not only supports competitive novel-view synthesis but can also serve as a latent space for generative models to generate 3D Gaussian scenes.
CoTL treats 3D Gaussian reconstruction as the objective for learning a compact scene representation, and its learned Token space can reuse mature sequence generative models (such as diffusion models and Transformer autoregression) to learn the distribution and generate new 3D Gaussian scenes.
Most novel-view synthesis methods assume that camera intrinsics and extrinsics are known, with poses pre-computed by SfM or SLAM; CoTL, however, directly takes pose-free multi-view images as input and needs to implicitly infer or absorb the geometric relationships between views.
Based on the abstract information, questions such as the performance of compact Tokens under extremely sparse views or large-scale scenes, the trade-off curve between Token count and reconstruction quality, and the specific metric comparisons on generative tasks still await further understanding.
All Facts (8)
CoTL (Compact Token Learning) proposes using a small set of compact latent Tokens to preserve the geometric and appearance information of a scene, allowing representation capacity to scale naturally with the number of observations.
50%UnverifiedCoTL's encoder takes a set of pose-free multi-view images as input, compresses each image into a small set of latent Tokens, and aggregates information across views to form a view-consistent scene representation.
50%UnverifiedIn CoTL's encoder, each additional view only requires extending the representation by a small portion of resolution-independent Tokens, allowing the representation's capacity to grow flexibly with the number of available observations rather than being limited by fixed-size network parameters.
50%UnverifiedCoTL uses a query-based Gaussian decoder that, through scene-agnostic learnable query vectors, accesses the Tokens produced by the encoder via an attention mechanism to recover a set of explicitly renderable 3D Gaussian primitives; novel-view rendering supervision signals train the entire compact representation end-to-end.
50%UnverifiedThe paper conducted experiments on datasets such as ACID, and the results show that CoTL's compact, unstructured Token set not only supports competitive novel-view synthesis but can also serve as a latent space for generative models to generate 3D Gaussian scenes.
50%UnverifiedCoTL treats 3D Gaussian reconstruction as the objective for learning a compact scene representation, and its learned Token space can reuse mature sequence generative models (such as diffusion models and Transformer autoregression) to learn the distribution and generate new 3D Gaussian scenes.
50%UnverifiedMost novel-view synthesis methods assume that camera intrinsics and extrinsics are known, with poses pre-computed by SfM or SLAM; CoTL, however, directly takes pose-free multi-view images as input and needs to implicitly infer or absorb the geometric relationships between views.
50%UnverifiedBased on the abstract information, questions such as the performance of compact Tokens under extremely sparse views or large-scale scenes, the trade-off curve between Token count and reconstruction quality, and the specific metric comparisons on generative tasks still await further understanding.
50%