Block Diffusion Models Challenge Autoregression: 1.43x Speedup for Longitudinal EHR Prediction

Autoregressive models excel at zero-shot prediction for longitudinal electronic health records (EHR), but their strictly sequential generation conflicts inherently with the way clinical events cluster and depend on one another over time. A new study proposes replacing autoregression (AR) with Block Diffusion (BD) and conducts a systematic comparison on the MIMIC-IV dataset, revealing the trade-offs among accuracy, efficiency, and representation quality.

Study Design and Core Findings

Under rigorous settings that match tokenization, context length, model capacity, and evaluation conditions, the research team compared BD against AR. BD partitions sequences into blocks, uses masked diffusion generation within blocks, and maintains an autoregressive or parallel structure between blocks. This design allows the model to denoise in parallel within a block, thereby breaking strict sequential dependencies.

Experiments show that BD exhibits a clear quality-efficiency trade-off across different block sizes: small-block configurations retain most of AR's predictive performance while significantly increasing inference speed. A representative configuration with B=8 achieves roughly 1.43x evaluation speedup. This result is directly appealing for clinical scenarios that require real-time or batch processing of large numbers of EHR sequences.

Masked Diffusion is a generation paradigm inspired by the masked pretraining idea from language models: during training, some tokens in the sequence are randomly masked, and the model learns to reconstruct the masked content based on the remaining context; during generation, it starts from a fully masked or partially masked state and progressively "denoises" toward the complete sequence over multiple iterations. Unlike autoregressive left-to-right token-by-token generation, each step of masked diffusion can predict tokens at multiple positions in parallel, so it naturally supports non-sequential generation. Block diffusion introduces a "block" granularity on top of this: masked diffusion is used for parallel denoising within blocks, while sequential dependencies can still be preserved between blocks, striking a compromise between generation-order flexibility and long-range consistency.

Separation of Generation Quality and Representation Learning

Although BD remains below AR in most settings for generative rollout, an important finding of the study is that frozen linear probes trained on BD and AR representations perform almost identically across various clinical prediction tasks.

This means that the clinical representations learned by BD are just as informative as those of AR. The gap in generation quality does not imply a gap in representation capability—for downstream prediction tasks, BD's representations can be used directly without bearing the accuracy loss of generative rollout. This separation provides a key argument for the practical deployment of BD: if the goal is classification or risk prediction rather than sequence generation, BD's efficiency advantages can be realized immediately.

A Frozen Linear Probe is a common method for evaluating representation quality: all parameters of a pretrained model are frozen, and only a simple linear classifier (or regression head) is trained on top of its output hidden representations, with performance evaluated on downstream tasks. Because the probe itself has very low capacity, its performance mainly reflects the task-relevant information that is already linearly separable in the representations, rather than the learning ability of the probe itself. Therefore, if two models have similar frozen linear probe performance, it indicates that the representations they learn are roughly comparable in information content, even if there is a gap in the quality of their generative rollout. This design decouples the questions of "are the representations good" and "does the generation look right," and is the key methodological foundation for the paper's conclusion that "a gap in generation quality does not equal a gap in representation capability."

Marginal Returns of Decoding Strategies

The study also examined the effects of increasing the denoising budget and alternative decoding strategies. Results show diminishing returns from increasing the number of denoising steps, and the gains from switching decoding strategies are also quite limited. This suggests that the performance bottleneck of the current BD configuration may not lie in compute investment during inference, but rather in the model architecture or training objective itself.

The authors point out that improving iterative decoding is a promising path to closing the remaining generation performance gap. In other words, BD's efficiency advantage is already established, but catching up with AR in generation quality requires smarter decoding algorithms rather than simply stacking more compute.

Generative Rollout refers to the inference mode in which a model starts from an initial state and progressively unfolds a complete sequence according to its own generation process. For autoregressive models, rollout means sequential token-by-token sampling; for diffusion models, rollout is the process of restoring a sequence from a noise or masked state through multi-step denoising. Rollout quality is typically measured by perplexity, sequence-level likelihood, or end-to-end performance on downstream tasks, and is affected by model architecture, training objective, and decoding algorithm. In this paper, "BD remains below AR in generative rollout" refers to the sequence quality gap during end-to-end generative unfolding, and this gap does not necessarily carry over to downstream prediction performance of frozen representations—the two need to be evaluated separately.

Implications for Longitudinal EHR Modeling

Longitudinal EHR data is characterized by events that occur in temporal clusters and are highly interrelated. The strict sequential generation assumption of AR models does not fully align with this characteristic, whereas the intra-block parallelism of block diffusion is closer to the actual structure of clinical events.

The value of this work lies in providing a clear trade-off framework: block size controls the balance point between quality and efficiency, with small blocks favoring efficiency and large blocks favoring quality. In clinical deployment, configurations can be selected according to specific task requirements—real-time alert scenarios favor small blocks, while offline analysis can tolerate larger blocks in exchange for quality.

The study also suggests that representation quality and generation quality can be evaluated in a decoupled manner. In the development of EHR foundation models, downstream prediction task performance does not necessarily need to come at the cost of generative rollout accuracy, which provides a more flexible dimension for model selection.

One-line Summary

Block diffusion achieves roughly 1.43x speedup on MIMIC-IV with small-block configurations, with representation quality comparable to autoregression.

Paragraph Summary

This study systematically compares block masked diffusion (BD) and autoregressive (AR) models on the MIMIC-IV dataset under matched tokenization, context length, model capacity, and evaluation conditions. BD partitions sequences into blocks, performs parallel denoising within blocks, and preserves sequential or parallel structure between blocks, thereby breaking strict sequential dependencies. Experiments show a clear quality-efficiency trade-off in BD: small-block configurations retain most of AR's predictive performance, with about 1.43x evaluation speedup at B=8. Although BD's generative rollout quality is mostly lower than AR's, frozen linear probes show that their representations are nearly identical on downstream clinical prediction tasks, indicating that generation quality and representation capability can be decoupled. In addition, increasing denoising steps yields diminishing returns and alternative decoding strategies offer limited gains, suggesting that the bottleneck lies in architecture or training objectives rather than compute investment, and that improving iterative decoding is the key direction for closing the gap.