MFGS: Feed-Forward Multi-View Feature Coding Redefines the 3D Gaussian Splatting Pipeline

3D Gaussian Splatting (3DGS) is transforming sparse-view 3D reconstruction, but existing feed-forward approaches generally adopt a two-stage "generate-then-compress" architecture, limiting coding efficiency. A new study proposes the MFGS framework, which embeds multi-view feature coding directly into the feed-forward pipeline, achieving up to 99.86% bitrate savings and roughly 12x end-to-end speedup across the RE10K, ACID, and DL3DV datasets.

Why "Generate-Then-Compress" Is a Bottleneck

The core advantage of feed-forward 3DGS methods is that, given sparse multi-view images, a single forward pass can reconstruct a 3D scene without per-scene optimization. This makes them highly attractive for real-time 3D content generation, immersive communication, and similar applications.

The problem lies in the transmission stage. Existing pipelines first generate a complete set of Gaussian representations (geometric attributes plus appearance attributes), then encode and compress each attribute separately. This "generate-then-compress" approach has two structural flaws:

First, the geometry and appearance encoders typically contain non-differentiable operations, preventing gradients from flowing smoothly during training and making end-to-end optimization of the entire pipeline difficult. Second, the bitstreams for different attributes are independent of each other, so cross-attribute redundancy cannot be exploited—information already implicit in the geometry may be redundantly stored again by the appearance encoder. The result is complex codec designs and limited rate-distortion performance.

In other words, reconstruction quality is not the issue—the transmission cost is locked in by the architecture itself.

Feed-Forward 3DGS: Traditional 3DGS requires tens of minutes of per-scene optimization, whereas feed-forward methods train a general network that takes sparse multi-view images as input and outputs Gaussian representations in a single forward pass, with no further optimization needed at inference time.

End-to-End Differentiable: This refers to the entire computation chain from input images to the final bitstream/rendered output being composed of differentiable operators, so that gradients from the loss function can backpropagate to the encoder network, enabling joint optimization of bitrate and distortion. Once non-differentiable modules such as quantization or entropy coding are inserted into the pipeline, approximation techniques like the straight-through estimator are typically needed to work around them, significantly increasing optimization difficulty.

Cross-Attribute Redundancy: Gaussian representations include multiple attributes such as position, covariance, spherical harmonics coefficients, and opacity. Geometry and appearance are physically strongly correlated (for example, object edges determine both geometry and color). If encoded separately, this correlation cannot be captured by a joint entropy model, resulting in wasted bits.

MFGS's Core Idea: Compress Features, Not Gaussians

MFGS (Multi-view Feature Coding for Gaussian Splatting) takes a straightforward approach: rather than compressing explicit Gaussian representations, it compresses the multi-view features extracted from input images.

Specifically, the encoder extracts compact features from multi-view images and, through end-to-end rate-distortion training, these features carry both geometric and appearance information. The decoder takes these features and directly predicts Gaussian attributes for rendering. The entire chain is differentiable, with joint optimization of bitrate and distortion.

This design yields two direct benefits:

• Cross-attribute redundancy is naturally exploited. Geometry and appearance are no longer two independent bitstreams but are unified at the feature level.
• Codec design is greatly simplified. There is no need to design complex compression modules for each attribute separately.

From an information-theoretic perspective, this moves the compression target from the "result" to the "intermediate representation," allowing redundancy to be eliminated at the source.

BD-rate (Bjontegaard Delta rate): A widely used metric in video/image compression that represents the average bitrate savings of a new method relative to a baseline at the same reconstruction quality. It is computed by integrating and comparing rate-distortion curves, avoiding the bias of comparing only a single operating point. A BD-rate saving on the order of 99.76%–99.86% is extremely rare in the compression field.

GMACs (Giga Multiply-Accumulate Operations): A common unit for measuring neural network inference computation—billions of multiply-accumulate operations. A 52%–55% reduction in total GMACs means the model requires nearly half the compute for the same task, which is critical for deployment on edge devices such as mobile phones or XR headsets.

Importance-Guided Gaussian Selection

Different rendering budgets require different numbers of Gaussians. MFGS introduces an importance-guided Gaussian selection mechanism that allows the receiver to flexibly adjust the number of Gaussians used for rendering from the same bitstream.

This is crucial for real-world deployment: when bandwidth is tight, render fewer Gaussians; when terminal compute is plentiful, use more—all without regenerating or re-encoding the bitstream. A single bitstream adapts to multiple terminal capabilities, reducing the overhead of preparing multiple bitstreams for different devices.

Experimental Data: Bitrate, Compute, and Speed

The research team validated on three datasets—RE10K, ACID, and DL3DV—with the following results:

• Bitrate: Measured by BD-rate, saving 99.76% to 99.86% compared to the "generate-then-compress" pipeline.
• Compute: Total GMACs reduced by 52% to 55%.
• Speed: Approximately 12x end-to-end speedup.

The bitrate savings are highly consistent across all three datasets, indicating that the gains come from architectural improvements rather than dataset-specific tuning. The 52%–55% compute reduction combined with 12x end-to-end speedup means this approach has deployment potential even on compute-constrained edge devices.

Implications for 3DGS Engineering

MFGS reveals a more general principle: in multi-view 3D reconstruction pipelines, compression should not happen after representation generation—it should happen in sync with representation learning.

When the feature learning objective includes a bitrate constraint, the model spontaneously learns an intermediate representation that is both easy to render and easy to compress. This is far more efficient than adding a compression module after the fact, and much easier to make end-to-end differentiable.

For engineering teams working on 3D reconstruction, immersive communication, or cloud rendering, the direct value of this work is that it demonstrates there is still an order-of-magnitude compression headroom in the transmission cost of feed-forward 3DGS—and that headroom lies in the pipeline architecture, not in optimizing individual codecs.

One-line Summary

MFGS embeds multi-view feature coding into the feed-forward 3DGS pipeline, achieving up to 99.86% bitrate savings and 12x speedup.

Paragraph Summary

Existing feed-forward 3DGS methods use a two-stage "generate-then-compress" architecture with independent encoding of geometry and appearance attributes, causing non-differentiable operations that block end-to-end training and preventing exploitation of cross-attribute redundancy—locking in transmission costs at the architectural level. The new MFGS framework embeds multi-view feature coding directly into the feed-forward pipeline: the encoder extracts compact features trained end-to-end with rate-distortion objectives to carry both geometric and appearance information, and the decoder predicts Gaussian attributes for rendering. Combined with importance-guided Gaussian selection, a single bitstream adapts to different rendering budgets. On RE10K, ACID, and DL3DV, MFGS achieves 99.76%–99.86% BD-rate savings, reduces total GMACs by 52%–55%, and delivers roughly 12x end-to-end speedup. The core insight: compression should happen in sync with representation learning, not after the fact on explicit Gaussian representations.