VAE

Stable Diffusion compresses the diffusion process from pixel space to latent space, implemented by a Variational Autoencoder (VAE).

Latent Diffusion通过VAE将图像从512×512×3的像素空间压缩到64×64×4的潜空间表示，数据量缩减约48倍

Stable Diffusion compresses a 512×512×3 image into a 64×64×4 latent representation via VAE, achieving a compression ratio of 48×.

The Latent Diffusion Model was proposed by Rombach et al. in 2022, combining the diffusion process with the latent space of a VAE, reducing the computational dimensions from 512×512×3 to 64×64×4.

David Ha and Schmidhuber published the paper "World Models" in 2018, demonstrating how an agent can complete reinforcement learning training within dreams

The VAE (Variational Autoencoder) is responsible for encoding and decoding between latent space and pixel space, and is one of the significant VRAM consumption points in video/image generation pipelines.

The latent diffusion architecture uses a Variational Autoencoder (VAE) to compress images into a low-dimensional latent space, performing noise addition and denoising in the compressed space, which is the origin of the name Stable Diffusion.

Bayesian inference in probability theory is the theoretical foundation of generative models such as VAEs and diffusion models.

The core innovation of VAE lies in encoding inputs as a probability distribution (typically a Gaussian distribution described by mean μ and variance σ) rather than a deterministic point, making the latent space continuous, smooth, and sampleable.

扩散模型在生成图像时本质上是在高维潜空间（Latent Space）中进行去噪采样，缺乏多角度约束时同一角色在不同帧中的特征易产生语义漂移

Stable Diffusion moves the diffusion process from pixel space to latent space, compressing images into low-dimensional representations via a VAE encoder before denoising, thereby reducing computational overhead.

Motion VAE is a class of methods that applies variational autoencoders to human motion sequence modeling; its encoder typically uses a Transformer or graph convolutional network, the decoder outputs a skeletal pose sequence, and the training objective is to maximize the ELBO.

A variational autoencoder (VAE) maps the input to a probability distribution (usually a Gaussian distribution), from which a latent vector is sampled and then the input is reconstructed, making the latent space more continuous and smooth and suitable for font interpolation.

The typical pipeline of font embedding includes: data preparation (collecting TTF/OTF font files and rendering characters), feature extraction (encoding into vectors with a CNN or VAE), training the embedding space (contrastive learning or autoencoder reconstruction loss), dimensionality reduction visualization (t-SNE, UMAP or PCA), and observing the structure.

The most direct benefit of abandoning the VAE is architectural simplification and a potential raise in the quality ceiling, while the main risk is that computational demands for training and inference may increase

The reconstruction quality of the VAE itself becomes the quality ceiling of the entire text-to-image system; if the VAE loses image details, the final output cannot recover that information

VAE (Variational Autoencoder) was proposed by Kingma and Welling in 2013

The VAE used by Stable Diffusion has been specifically optimized with perceptual loss and adversarial training, and its reconstruction quality far surpasses that of early standard VAEs, but issues such as high-frequency detail loss and color shift still remain

Normalizing flows stand in contrast to VAEs (which require variational lower-bound approximation) and GANs (which cannot compute likelihoods)

A variational autoencoder (VAE) consists of an encoder and a decoder; its latent space has continuity and interpolability, making it suitable for generative tasks.