SCION: Reconstructing 3D Scene Representations with Reusable Neural Primitives

Real-world scenes are inherently compositional — bricks, blades of grass, pebbles, and leaves recur repeatedly across both man-made and natural environments. Yet mainstream neural scene representation methods largely ignore this repetition, treating every element as a unique individual to be modeled from scratch. SCION (Scene Composition with Instanced Neural Primitives), proposed by a team at Princeton University, directly addresses this pain point — training neural scene representations to "discover reusable parts" rather than memorizing an entire scene element by element.

The Redundancy Problem in Existing Methods

Current approaches represented by 3D Gaussian Splatting (3DGS) and its downstream abstraction and compression variants universally treat every element in a scene as a unique entity, fitting millions of independent Gaussians to a single scene. This creates two direct problems: first, enormous amounts of redundant parameters are stored, with similar blades of grass or similar bricks each occupying their own independent parameter space; second, the representations offer weak handles for downstream manipulation — when you want to edit or animate a class of repeating elements, there's no effective "grip" to grab onto.

Prior methods like Splat and Replace have attempted to fit template objects for structural reuse, but they rely heavily on manual selection of repeating elements, making them low in automation and difficult to scale. This means existing solutions have never found an elegant balance between compression efficiency and controllability.

3D Gaussian Splatting (3DGS) is a real-time neural scene rendering method introduced in 2023. Its core idea is to represent a scene's geometry and appearance using a large collection of 3D Gaussian ellipsoids (Gaussian splats). Each Gaussian carries attributes including position, rotation, scale, opacity, and spherical harmonic coefficients (for view-dependent color). These are rendered into images through a differentiable rasterization process and compared against real photographs to optimize all parameters. 3DGS excels in rendering speed and reconstruction quality, but a typical scene often requires millions to tens of millions of Gaussians, with per-scene storage reaching hundreds of MB — making compression and structured representation central research topics in this area.

SCION's Core Idea: Vocabulary Plus Instantiation

SCION proposes a hierarchical, compositional scene representation. Its core insight is to replace mutually independent Gaussians with a compact vocabulary of reusable primitives, paired with lightweight world-space instances that place transformed copies of these primitives throughout the scene.

In other words, the scene is no longer a pile of millions of isolated Gaussians, but rather "a set of templates + a set of placement instructions." Templates define the recurring geometric and appearance units; instances record the position, rotation, scale, and other transformation parameters of each copy in world coordinates. This structure naturally aligns with the compositional nature of the real world and dramatically reduces parameter count.

Joint Optimization of Discrete and Continuous Parameters

To recover this structure from multi-view captured data, SCION employs a joint optimization strategy over both discrete and continuous parameters. It combines two-level densification at both the splat and instance levels, and introduces an adversarial loss to preserve detail across shared primitives — because once multiple instances share a single template, the critical challenge becomes preventing details from being averaged out and blurred. The adversarial loss is introduced precisely to maintain high-quality visual fidelity.

Densification is a key mechanism in the 3DGS training pipeline: during optimization, the system adaptively splits or clones Gaussians in regions with high reconstruction error to improve local detail expressiveness. SCION extends this mechanism to two levels — the splat level (the Gaussian distribution within a primitive) and the instance level (the number and placement of primitive copies in the scene) — allowing the system to increase representational capacity where more detail is needed, while maintaining template sharing. The adversarial loss draws from the GAN training paradigm: a discriminator distinguishes "genuine detail" from "detail blurred by multi-instance averaging," forcing primitive templates to maintain sharp high-frequency textures under the shared constraint, rather than degenerating into a blurry average of all instance appearances.

Performance and Practical Value

As described in the paper, the structure recovered by SCION is both compact and controllable. Even compressed to as little as 1.2 MB, it maintains high rendering quality. In terms of rate-distortion performance, it outperforms existing Gaussian compression methods — meaning it delivers better image quality at the same storage budget, or uses less storage at the same quality level.

More practically valuable is that this representation naturally supports instance-level scene editing and animation without retraining. Because the scene is decomposed into reusable templates and instance transforms, users can directly manipulate a category of elements — for example, uniformly adjusting the pose of all leaves, or animating a class of objects — without needing to re-optimize the entire scene as traditional methods require.

Rate-distortion is the core trade-off framework in information theory and compression: "rate" refers to the number of bits required to store or transmit a representation, while "distortion" refers to the degradation in reconstruction quality after compression (commonly measured by PSNR or SSIM). An ideal compression method should minimize distortion at a given bit budget, or minimize bit rate at a given quality target. In the neural scene representation domain, rate-distortion curves have become a standard tool for evaluating compression methods. SCION claims to outperform existing Gaussian compression baselines across this curve, suggesting that its structured compositional representation itself constitutes an efficient form of implicit compression — rather than relying on post-processing quantization or entropy coding.

Implications for Neural Scene Representation

SCION conveys an inspiring idea: neural scene representations don't need to rote-memorize a scene as a pile of independent primitives — they can actively "discover" reusable compositional parts within a scene. This approach more closely mirrors how humans understand the world: we recognize scenes by identifying recurring object categories, not by memorizing pixel by pixel.

From an engineering perspective, this compositional representation simultaneously benefits compression, editing, and animation across multiple dimensions, and may represent a notable direction for technical evolution beyond 3DGS. The project page is publicly available at Princeton's Computational Imaging Lab (light.princeton.edu/SCION) for researchers who wish to explore the technical details further.

It should be noted that this article is based on a summary of an arXiv preprint. The specific experimental comparisons, ablation analyses, and generalization capabilities remain to be further verified by the full paper and open-source implementation.