Lafa: An 8B-Parameter Continuous Flow Language Model with 2.54x Throughput Over AR Models

Continuous flow language models offer control and acceleration tools for parallel generation that discrete diffusion models lack, but effective techniques for building them at the billion-parameter scale remain immature. Recent research introduces Lafa (Large flow language model), a data-efficient, high-performance flow language model scaled to 8B parameters.

Architecture Design: Block-wise Flow and Asynchronous Generation

Lafa adopts a block-wise flow architecture, using causal attention for the prefix and bidirectional, asynchronous flow within each block. This design brings two key advantages: during training, efficient training can be achieved by adapting pretrained autoregressive models; during inference, fast generation can be achieved using prefix caching and block-wise parallel decoding.

Unlike discrete diffusion models, continuous flow models allow bidirectional modeling within blocks while maintaining causality of the prefix. This hybrid attention pattern preserves the autoregressive model's ability to precisely model context while introducing the flexibility of parallel generation.

Block-wise Flow and Asynchronous Generation: Block-wise flow is a generation paradigm that partitions the sequence into fixed-size blocks. Within each block, continuous flow modeling is used, i.e., gradually transforming from noise to the data distribution via ordinary differential equations (ODE) or flow matching; between blocks, causal order is maintained, and the causal attention of the prefix ensures that the current block can only see information from previous blocks. Asynchronous generation means that positions within a block can be updated in parallel, rather than strictly generating token by token from left to right. This design allows inference to reuse computed prefix representations via prefix caching and supports parallel decoding within blocks, significantly reducing latency. Unlike discrete diffusion models, continuous flow allows bidirectional dependency modeling within blocks while maintaining autoregressive-style context control through inter-block causality, providing more flexible acceleration tools for parallel generation.

Training Efficiency: Multi-scale Training Completed Within 30B Tokens

The research team trained Lafa at three scales: 0.5B, 7B, and 8B, each using fewer than 30B tokens. This data efficiency significantly outperforms previous diffusion models—which typically require hundreds of billions of tokens to achieve comparable performance.

The training process starts from a pretrained autoregressive model and adapts it through flow training. This transfer learning strategy not only saves computational resources but also brings an unexpected benefit: flow training, through the multi-token prediction effect, preserves or even improves the capabilities of the pretrained AR model. This means the model gains parallel generation capabilities without sacrificing its original language understanding and generation quality.

Flow Training and Multi-token Prediction Effect: Flow training is a method for adapting a pretrained autoregressive model into a continuous flow model. Through a flow matching objective, it learns a continuous transformation process from a simple noise distribution to the language data distribution. During training, the model is required to predict tokens at multiple positions within a block, which naturally produces a multi-token prediction effect—the model learns not only the next token but also tokens at other positions within the block. This dense prediction signal helps the model capture richer local dependencies, thereby preserving or even improving the language modeling capability of the pretrained autoregressive model during adaptation. The multi-token prediction effect is also the basis for draft generation in self-speculative decoding, because the model can produce multiple candidate tokens at once for subsequent verification.

Performance: Matching AR Quality, Distillation Enables Few-step Generation

Evaluations on instruction following, reasoning, and coding tasks show that the flow model learned by Lafa matches or even exceeds the quality of autoregressive models. More importantly, the flow model can be distilled into flow maps to achieve single-step or few-step generation, further reducing inference latency.

This distillation capability provides flexibility for practical deployment: use few-step generation in scenarios requiring extreme speed, and use full flow sampling in scenarios where quality is prioritized.

Flow Maps and Few-step Generation: Flow maps are a distillation form of continuous flow models, aimed at compressing the flow sampling process that originally requires multi-step numerical integration (such as Euler's method) into a single-step or few-step mapping. Specifically, flow maps learn a deterministic transformation directly from noise to data, bypassing iterative solving and thus significantly reducing inference latency. In Lafa, the flow model can be distilled into flow maps to achieve single-step or few-step generation while maintaining generation quality as much as possible. This distillation capability provides flexibility for practical deployment: use few-step generation in scenarios requiring extreme speed, and use full flow sampling in scenarios where quality is prioritized. The introduction of flow maps makes continuous flow models more competitive in inference efficiency and expands their application scope.

Self-Speculative Decoding: 2.54x Throughput Improvement Within a Single Model

Leveraging the multi-token prediction effect brought by flow training, the research team designed a self-speculative decoding scheme: within a single model, flow-based drafts are used for autoregressive verification. Unlike standard speculative decoding that requires an independent draft model, this method does not require additional model parameters or memory overhead.

Experimental results show that, at comparable quality, this scheme achieves 2.54x the throughput of autoregressive models, outperforming multi-token prediction and discrete diffusion methods. This result indicates that continuous flow can serve not only as a generation paradigm but also as a practical tool for accelerating existing AR models.

Self-Speculative Decoding: Speculative decoding is a technique for accelerating autoregressive generation, typically using an independent draft model to quickly generate multiple candidate tokens, which are then verified in parallel by the main model, thereby improving throughput while maintaining output quality. Standard speculative decoding requires an additional draft model, increasing parameter and memory overhead. Self-speculative decoding implements draft generation and verification within the same model: leveraging the multi-token prediction capability brought by flow training, the model can quickly generate multiple candidate tokens based on flow drafts and then verify these candidates in parallel in an autoregressive manner. Since the draft and verification share the same set of parameters, no additional model is needed, saving memory and computational resources. Lafa's experiments show that this scheme achieves 2.54x the throughput of autoregressive models at comparable quality, outperforming multi-token prediction and discrete diffusion methods, indicating that continuous flow can serve not only as a generation paradigm but also as a practical tool for accelerating existing AR models.

Significance and Limitations

Lafa's findings show that continuous flow can serve as a scalable foundation for language modeling. Its data-efficient training, compatibility with pretrained AR models, and the acceleration effect brought by self-speculative decoding pave the way for the practical application of flow language models.

However, current research is still concentrated at the 8B parameter scale, and performance at larger scales remains to be verified. In addition, the impact of hyperparameters in the block-wise flow architecture (such as block size and number of flow steps) on performance has not yet been systematically analyzed. The scalability of flow models on long-context and complex reasoning tasks is also a direction for future exploration.

One-line Summary

Lafa is a data-efficient, 8B-parameter block-wise flow language model that matches AR quality and achieves a 2.54x throughput improvement.

Paragraph Summary

Lafa is a block-wise flow language model that uses causal attention for the prefix and bidirectional asynchronous flow within blocks, supporting prefix caching and block-wise parallel decoding. The research team trained at three scales—0.5B, 7B, and 8B—using fewer than 30B tokens per scale, significantly outperforming previous diffusion models in data efficiency. Flow training preserves or improves the capabilities of pretrained autoregressive models through the multi-token prediction effect and supports distillation into flow maps for few-step generation. On instruction following, reasoning, and coding tasks, Lafa's quality matches or even exceeds that of autoregressive models. Self-speculative decoding, designed using the multi-token prediction effect, performs autoregressive verification with flow drafts within a single model, achieving 2.54x the throughput of AR models without additional model parameters. The research shows that continuous flow can serve as a scalable foundation for language modeling, but scalability at larger scales and long contexts still needs verification.