The Rise of Diffusion Language Models: DiffusionGemma Crushes Autoregressive Models in Speed Tests

Introduction: A Battle Over New Paradigms in Language Models

For a long time, the mainstream architecture of large language models (LLMs) has been the Transformer-based autoregressive generation approach—predicting the next word one token at a time. Since Google published its "Attention Is All You Need" paper in 2017, the Transformer architecture has revolutionized the NLP field through its Multi-Head Self-Attention mechanism. Autoregressive language models (such as the GPT series) build on this by introducing causal masking, allowing the model to attend only to tokens preceding the current position, thereby enabling left-to-right sequential generation. However, a comparative test focused on generation speed has once again thrust Diffusion Language Models into the spotlight of community discussion.

A Reddit user conducted a head-to-head comparison of generation speed between DiffusionGemma and Deepseek Flash, and the results left them "completely stunned by how fast the diffusion model was." This real-world feedback reignited a deeper industry debate about the underlying generation paradigms of language models.

Autoregressive vs. Diffusion: Two Fundamentally Different Generation Philosophies

The Inherent Speed Bottleneck of Autoregressive Models

Autoregressive models—represented by the Deepseek and GPT series—employ strict "left-to-right" sequential generation. Each time a token is generated, all preceding tokens must be re-fed into the model as context for a forward pass computation.

From a mathematical standpoint, autoregressive models decompose the joint probability P(x₁,x₂,...,xₙ) into a product of step-by-step conditional probabilities (i.e., the chain rule: P(x₁,...,xₙ) = ∏P(xᵢ|x₁,...,xᵢ₋₁)), using a Causal Attention Mask to ensure each token can only "see" the historical information on its left. This design allows parallel loss computation during training via "teacher forcing" techniques, but requires strictly serial execution during inference—each generation step depends on the previous step's output as its new input condition, forming an unbreakable chain of sequential dependencies.

This mechanism yields stable generation quality and logical coherence, but its drawback is equally evident: the generation process is serial and cannot be parallelized. When the output text is long, the number of tokens directly determines inference time, forming a natural upper limit on the speed of autoregressive models.

At the engineering level, as sequence length increases, the KV Cache grows linearly, and memory bandwidth becomes the hardware ceiling constraining speed. The KV Cache avoids redundant computation by caching already-computed Key and Value tensors in memory—for a typical 7B model with 32 layers and 32 attention heads, each token requires about 0.5MB of memory to store its KV Cache, so generating 1000 tokens requires an additional ~500MB of memory. During large-batch concurrent inference, this pressure becomes the core hardware bottleneck limiting service throughput, and is the direct impetus behind the emergence of engineering optimizations such as FlashAttention and PagedAttention. Even versions branded as "Flash" (fast) are, in essence, still constrained by this token-by-token generation architecture.

The Parallel Generation Advantage of Diffusion Models

Diffusion models take an entirely different approach. Drawing on the successful experience of the image generation field (such as Stable Diffusion), they treat text generation as a "noise-to-clarity" denoising process. The model can predict and iteratively refine all tokens of the entire sequence in parallel, rather than outputting them one by one.

Diffusion models first rose to prominence in image generation, represented by DDPM (Denoising Diffusion Probabilistic Models, 2020), whose core idea is to progressively recover data from Gaussian noise by learning a Reverse Diffusion Process. It's worth noting that transferring image diffusion to text is no easy task. Image pixels belong to a continuous data domain, whereas text tokens are discrete and cannot be directly interpolated continuously. Academia has primarily solved this challenge along two paths: first, performing the diffusion process in a continuous Embedding Space (such as Diffusion-LM), mapping tokens to continuous embedding vectors before performing diffusion, then doing nearest-neighbor decoding after generation; second, defining a discrete Markov transition matrix to perform masking and replacement operations directly in token space (such as D3PM and MDLM), where the forward process progressively masks or randomly replaces tokens, and the reverse process learns to predict the original masked tokens. The discrete diffusion path adopted by DiffusionGemma is precisely the key technical choice behind recent quality breakthroughs in text diffusion models.

This is the core reason for DiffusionGemma's astonishing speed—diffusion models complete generation through a fixed number of iteration steps, with a computational pattern closer to the batched inference of image generation, fully leveraging the parallel computing power of GPUs. Moreover, these steps do not scale linearly with output length; in other words, whether outputting 100 tokens or 500 tokens, the increase in time for diffusion models grows far more gently than for autoregressive models.

The Technical Implications Behind the Speed Tests

The Exploratory Significance of DiffusionGemma

DiffusionGemma is an experimental project that combines Google's Gemma model architecture with the diffusion generation paradigm, representing a noteworthy trend: the open-source community is attempting to transfer the capabilities of mature foundation models onto more efficient generation frameworks.

Based on real-world feedback, the diffusion paradigm's advantage in inference latency is highly significant. In engineering evaluations, inference performance is typically characterized by two core metrics: Time To First Token (TTFT) measures the wait time from request to the appearance of the first output token, directly affecting the user's perceived responsiveness; Throughput, measured in tokens/second, gauges the model's sustained generation rate, determining the total time for long-text tasks and the concurrent processing capacity of the service. An autoregressive model's throughput is constrained by token-by-token serial decoding; because diffusion models predict the entire sequence in parallel, their performance characteristics are closer to "one-time batch processing," with a particularly prominent throughput advantage in long-sequence scenarios. For low-latency response scenarios—such as real-time conversation, code completion, and batch text processing—this speed improvement carries direct commercial value.

Capability Tradeoffs That Warrant a Rational View

Speed is not the only dimension for judgment. Current diffusion language models still face several unresolved challenges:

• Generation quality: Whether diffusion models can fully match mature autoregressive models on tasks such as complex reasoning and long-text coherence still requires more benchmark validation.
• Controllability: Autoregressive models have natural advantages in instruction following and step-by-step reasoning (such as Chain-of-Thought, CoT). Chain-of-Thought (CoT), proposed by Google in 2022, improves the accuracy of complex reasoning by guiding the model to generate intermediate reasoning steps before outputting the final answer. CoT's effectiveness is highly coupled with the sequential generation mechanism of autoregression: when generating "step N+1," the model can use the output of "step N" as context, enabling genuine step-by-step logical reasoning. The parallel global generation mechanism of diffusion models, however, faces a structural challenge—theoretically, it needs to predict subsequent content simultaneously without having determined the preceding reasoning steps, which may reduce the logical self-consistency of the reasoning process. Researchers are currently exploring methods such as "Chunk-wise Diffusion" to mitigate this issue—splitting the sequence into several ordered chunks, maintaining dependencies between chunks while diffusing in parallel within chunks—but a mature solution has yet to emerge.
• Ecosystem maturity: Autoregressive models have well-developed toolchains, fine-tuning methods, and deployment optimization solutions, whereas the supporting ecosystem for diffusion language models is still in its early stages.

Therefore, while purely speed-based comparisons are exciting, a complete capability assessment still requires a comprehensive evaluation across multiple dimensions including quality, cost, and stability.

The Future Prospects of Diffusion Language Models

From Inception Labs' Mercury to various research-oriented diffusion LLMs, industry investment in this direction has clearly increased in recent years. Its core appeal lies in this: at a time when inference cost continues to be a key bottleneck for AI deployment, any architectural innovation that can substantially reduce latency and compute consumption could reshape the competitive landscape.

The Mercury series of models released by Inception Labs is the most commercialized representative of diffusion language models to date, claiming to achieve generation speeds exceeding 1000 tokens/second on code generation tasks. Notably, Mercury does not pursue pure fully-parallel diffusion, but instead adopts a "Semi-Autoregressive Diffusion" design—splitting the output sequence into multiple fixed-size chunks, performing diffusion denoising on all tokens within a chunk in parallel, while generating chunks sequentially in autoregressive order. The engineering logic behind this compromise architecture is that pure fully-parallel diffusion suffers quality loss in long-text coherence and logical dependency handling, whereas introducing autoregressive dependencies between chunks can significantly improve output quality and controllability at the cost of only a small amount of parallelism. Mercury's success validates the feasibility of the "hybrid architecture" route and provides an important reference point for exploratory projects like DiffusionGemma.

If diffusion models can gradually approach the generation quality of autoregressive models while maintaining their speed advantage, we may well be on the eve of a paradigm shift in language model generation. Exploratory projects like DiffusionGemma are important pioneers of this transformation.

Conclusion

Although this speed test from the Reddit community originated from an individual user's intuitive experience, it reflects a thought-provoking technical signal: autoregression is no longer the only answer for language models. The diffusion paradigm, with the speed advantage brought by parallel generation, is carving out a brand-new path for efficient AI inference.

For developers, now may be a good time to pay attention to and experiment with diffusion language models—especially in application scenarios with high demands on response speed. Of course, the ultimate technical choice should still be built on a comprehensive tradeoff between speed and generation quality, rather than a single-dimensional test result.