SplatQ: Reshaping 3D Gaussian Splatting Pruning with Quadratic Optimization

The Pruning Dilemma in 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) has emerged as a leading technique in novel view synthesis, offering high-quality rendering with real-time performance. However, the sheer number of Gaussian primitives introduces significant storage and computational overhead, making pruning an essential optimization step.

Most existing pruning methods follow a straightforward approach: assign an independent importance score to each Gaussian, then perform a global Top-K selection to retain the highest-scoring ones. While seemingly reasonable, this approach harbors two fundamental structural flaws.

First, it lacks regional budget control. Global Top-K selection allows regions with consistently high importance scores to consume a disproportionate share of the budget, while detail-rich regions with relatively lower scores get over-pruned — leading to an imbalanced reconstruction quality across the scene. Second, it ignores joint deletion effects. Gaussians don't exist in isolation; removing multiple interrelated Gaussians simultaneously produces a cumulative impact far greater than the sum of their individual effects.

The fundamental principle of 3DGS is to represent a scene using a set of 3D Gaussian ellipsoids, where each Gaussian primitive carries attributes such as position, rotation, scale, opacity, and spherical harmonic coefficients (for view-dependent color). During rendering, these Gaussian primitives are "splatted" onto a 2D image plane and composited through alpha blending after depth sorting. Compared to NeRF's dense ray-marching sampling, 3DGS's rasterization pipeline is better suited for GPU parallelism, enabling real-time frame rates. However, a typical scene may contain millions of Gaussian primitives, consuming large amounts of VRAM (often hundreds of MB to several GB), making direct deployment on mobile or embedded devices nearly impossible. Pruning has therefore become one of the key bottlenecks for 3DGS in real-world applications.

SplatQ's Combinatorial Pruning Approach

To address these issues, researchers propose SplatQ — a combinatorial optimization pruning algorithm that coordinates regional budget allocation and joint Gaussian selection under a fixed budget. Its core innovation lies in transforming pruning from "independent scoring, global ranking" to "partitioned coordination, joint solving."

Spatial and Appearance Partitioning

SplatQ first divides all Gaussians into size-constrained blocks based on spatial proximity and appearance similarity. The key benefit of this step is decomposing the otherwise intractable global selection problem into manageable subproblems. Partitioning not only reduces optimization complexity but also provides natural units for subsequent regional budget control.

Budget Quota Smoothing

After partitioning, SplatQ computes retention quotas for each block based on its importance, then applies smoothing to these quotas. The purpose is straightforward: prevent budgets from concentrating in a handful of high-importance blocks, ensuring reasonable Gaussian retention across all parts of the scene. Ablation studies confirm that budget smoothing significantly improves the reconstruction of local details — precisely where traditional global Top-K falls short.

Quadratic Approximation for Modeling Joint Effects

The most technically substantial component of SplatQ is its quadratic approximation model for joint deletion effects. It leverages the response of representative pixel colors to changes in Gaussian opacity to construct a quadratic approximation model. This model incorporates three types of factors: the independent response of individual Gaussians, pairwise interactions between Gaussians, and quota penalty terms.

These factors are integrated into a Quadratic Unconstrained Binary Optimization (QUBO) objective function. QUBO is a mathematical form naturally suited for expressing combinatorial selection problems — each Gaussian's "keep or delete" decision corresponds to a binary variable, while pairwise interactions are captured by quadratic terms.

Quadratic Unconstrained Binary Optimization (QUBO) is a standard problem formulation in combinatorial optimization: the objective function consists of linear and quadratic cross terms of binary variables (0 or 1), with no equality or inequality constraints. Its mathematical form minimizes $x^T Q x$, where $x$ is a binary vector and $Q$ is a real-valued matrix. QUBO is mathematically equivalent to the Ising Model — a physical model describing the energy of a magnetic spin system, where spin states correspond to ±1 binary variables. This equivalence allows quantum annealers (such as D-Wave) and Coherent Ising Machines to directly "compute" QUBO problems without conventional digital logic, offering theoretical speed advantages at certain problem scales. SplatQ's encoding of Gaussian "keep or delete" decisions as QUBO represents a concrete attempt to bring physics-inspired computing into graphics optimization.

Solving with Annealing and Ising Machines

Once the QUBO objective is constructed, SplatQ employs two solving strategies: Simulated Annealing (SA) and Coherent Ising Machines (CIM).

Simulated Annealing is a classic heuristic optimization algorithm well-suited for finding near-optimal solutions on conventional hardware. The inclusion of Coherent Ising Machines is notably forward-looking — these specialized physical computing devices are designed specifically to solve Ising model and QUBO problems, meaning SplatQ's framework can run on traditional computing platforms while also leaving a door open for future integration with next-generation optimization hardware.

This choice of solver reflects an added dividend from translating a graphics problem into a combinatorial optimization problem: the ability to directly leverage mature optimization toolchains and emerging specialized computing hardware.

Coherent Ising Machines (CIM) are physical computing devices based on networks of optical parametric oscillators, originally co-developed by Stanford University and NTT Research in Japan. They map variables in a QUBO problem to the phase states of optical pulses (0-phase or π-phase corresponding to 0/1), and use measurement-feedback mechanisms within optical networks to simulate the evolution of Ising spins toward the lowest-energy state (i.e., the optimal solution). CIMs demonstrate superior solution quality over conventional simulated annealing for fully-connected graph problems, particularly in scenarios with large numbers of pairwise variable interactions — which closely matches the pairwise Gaussian interaction structure in SplatQ. Currently, CIMs are primarily accessible via cloud services with latency and scale limitations, but SplatQ's inclusion of them as a solver option demonstrates a viable pathway for connecting 3DGS optimization with emerging computing paradigms.

Experimental Results and Significance

The team evaluated SplatQ on 5 benchmark datasets. Results show that SplatQ outperforms current state-of-the-art pruning methods across three key dimensions: aggregate reconstruction quality, spatial quality balance, and rendering speed.

Ablation studies further decompose the contributions of the two core design choices — budget smoothing is responsible for improving the reconstruction of local details, while interaction modeling significantly improves the overall quality of Gaussian selection under a fixed budget. The two components are complementary and together underpin SplatQ's performance advantages.

From a methodological perspective, SplatQ's significance extends beyond proposing a better pruner. It demonstrates a pathway for reformulating 3DGS optimization problems as QUBO combinatorial optimization problems, formally incorporating the long-neglected "joint deletion effects" and "regional budget allocation" into the optimization objective. For 3DGS applications that prioritize storage efficiency, rendering speed, and quality balance, this approach offers a new and replicable framework.

Summary

SplatQ upgrades 3DGS pruning from a coarse, independent-scoring paradigm to a refined combinatorial optimization approach that accounts for interactions and budget constraints. Through spatial-appearance partitioning, quota smoothing, quadratic approximation modeling, and QUBO solving, it achieves comprehensive improvements in reconstruction quality, spatial balance, and rendering speed — while also opening up possibilities for integration with next-generation computing hardware such as Coherent Ising Machines.