EquivFlow: VAE-Aware Encoding to Fix Dark-Side Collapse in SDR-to-HDR Video

Why SDR-to-HDR Is a Hard Problem

Reconstructing standard dynamic range (SDR) video into high dynamic range (HDR) video is a classic ill-posed problem. In the original SDR content, highlights are clipped, shadows are crushed, and quantization artifacts are mixed in. The reconstruction process needs to recover these lost details while faithfully preserving the existing content. Latent diffusion models, with their powerful generative priors, provide a strong tool for restoring missing details, but their effectiveness depends heavily on how the HDR signal passes through a pretrained VAE (variational autoencoder).

A recent study identified the key bottleneck: common log-like encoding suppresses low-luminance chroma differences before generation, causing "dark-side collapse" in the frozen VAE latent space. In other words, color information in dark regions is already lost before entering the diffusion model, and no amount of generative power can recover it.

Ill-posed problem: In mathematics and signal processing, an inverse problem is ill-posed when the solution cannot be uniquely determined from the observed data alone. SDR-to-HDR is exactly this: highlight clipping means luminance information above the upper limit is permanently discarded, shadow crushing blends details below the threshold with noise, and quantization artifacts further contaminate the remaining signal. A single SDR pixel may correspond to countless HDR original scenes, and the reconstruction algorithm must make choices without constraints. Diffusion models work because their generative prior acts as a strong constraint—it "knows" what natural images and videos typically look like, thereby selecting the most plausible solution among many feasible ones. But this also means that the quality of the prior and the quality of the input representation jointly determine the final result; neither can be missing.

The role of VAE (variational autoencoder) in diffusion models is often underestimated. Latent diffusion models such as Stable Diffusion do not denoise directly in pixel space; instead, they first use the VAE encoder to compress the image into a low-dimensional latent space, complete the diffusion process there, and finally restore pixels with the decoder. The encoder of a pretrained VAE is frozen, and it determines which information can enter the generation stage and which is discarded. If the encoding step itself compresses away dark-region chroma, then no matter how powerful the diffusion model is, it cannot conjure this information back in latent space. This is precisely the mechanism of "dark-side collapse"—the bottleneck is not the generator, but the mismatch between the encoder and the HDR signal representation.

EquivFlow: Equivariant Exposure Encoding Without Modifying the VAE

To address this problem, researchers proposed EquivFlow—a compact, invertible exposure-equivariant encoding. Its core design maps exposure changes to an additive shift while preserving a scale-independent chroma component, without any modification to the pretrained VAE.

The clever part of this design is the word "equivariant": when the exposure of the input video changes, the encoded representation changes accordingly in a predictable additive way, rather than nonlinearly compressing dark-region chroma as log encoding does. The chroma component is designed to be scale-independent, meaning it is not lost when brightness is scaled, thereby preserving complete color information in dark regions.

Equivariance is a core concept in group theory and geometric deep learning, contrasted with invariance. Invariance means the output remains unchanged after an input transformation, while equivariance means the output transforms in a predictable, structured way in sync with the input transformation. Take exposure as an example: if the overall scene brightness is increased by one stop, an ideal encoding should add a fixed offset accordingly, rather than nonlinearly flattening the dark regions. This property allows the model to maintain consistency at the representation level without explicitly learning exposure changes. Equivariant encoding has been widely used in physics-inspired neural networks, and EquivFlow introduces it into HDR representation, essentially enabling the encoder to "understand" exposure as an additive transformation, thereby avoiding the nonlinear suppression of dark-region chroma by logarithmic curves.

Encoding-Level Validation: Robustness Across VAEs

The researchers conducted encoding-level tests on multiple pretrained image and video VAEs. The results show that EquivFlow's exposure structure and deep-dark-region chroma transmission characteristics are preserved across all tested hosts. This indicates that the encoding scheme is not overfitted to a specific VAE but is general across model architectures.

This is especially important for practical deployment: different video generation pipelines may use different VAE backbones, and if an encoding scheme only works on a single model, its application value is greatly reduced. EquivFlow's cross-host consistency supports its broad applicability.

Results When Integrated into a Video Diffusion Pipeline

After integrating EquivFlow into the SDR-to-HDR video diffusion pipeline, this method delivered consistent chroma fidelity improvements across various tested settings while maintaining competitive reconstruction fidelity. This means dark-region colors no longer collapse, and overall image quality is not compromised as a result.

You may not have noticed that this improvement is achieved without modifying the frozen VAE; it is a lightweight modification at the encoding level. For diffusion model pipelines already in use, this approach avoids the enormous cost of retraining or fine-tuning the VAE, offering good engineering feasibility.

Implications for HDR Reconstruction Practice

The value of this work lies not only in proposing a specific encoding, but also in revealing an easily overlooked link: how the HDR signal is represented before entering the generative model may be more critical than the generative model itself. Log encoding is widely used in the HDR field, but its suppression effect on low-luminance chroma had not been given sufficient attention before.

For developers working on video restoration, SDR-to-HDR, and diffusion-based image enhancement, EquivFlow's approach suggests that before tuning the generative model, first checking whether there is an information bottleneck in the data encoding step can often yield more obvious gains at lower cost.

Why log encoding became popular in HDR and why it is a hidden risk here: The human eye's perception of brightness approximates a logarithmic relationship, and HDR dynamic range spans multiple orders of magnitude. Logarithmic transforms can compress the wide range into an interval that networks can easily handle, which is why HDR transfer functions such as PQ and HLG, as well as many preprocessing pipelines, use log-like curves. The problem is that this compression of brightness is nonlinear: in low-luminance regions, small chroma differences between adjacent pixels are further reduced, even below quantization precision, and are thus irreversibly lost before entering the VAE. In other words, log encoding optimizes the representation efficiency of luminance dynamic range at the cost of dark-region chroma distinguishability. The value of EquivFlow is that it reminds developers: the choice of encoding curve is not a neutral preprocessing step, but a key design decision that determines where the information bottleneck lies.