AnchorDLM: Anchor Mechanism Enhances Controllability and Reasoning of Diffusion Language Models

Diffusion Language Models (DLMs) are emerging as a new direction for controllable text generation. Unlike the token-by-token generation paradigm of autoregressive models, DLMs generate entire passages of text through bidirectional attention and iterative denoising. This characteristic allows "control information" to intervene from any position in the sequence, theoretically granting stronger controllability. But theoretical possibility does not equal practical effectiveness—whether the content of control tokens matches their placement directly determines whether guidance succeeds. A recent study proposed the AnchorDLM framework, which attempts to jointly optimize the two questions of "where to place control information" and "what content to place," and validated its effectiveness on code generation and mathematical reasoning tasks.

The Controllability Dilemma of Diffusion Language Models

Control in autoregressive models typically relies on prefix prompts or prefix fine-tuning, where control signals can only be applied from the beginning of the sequence. The bidirectional attention structure of DLMs breaks this limitation: control tokens can be inserted at any position in the output, "pulling" the generation process from the middle.

Problems follow. The paper points out that effective guidance depends on the compatibility between the content of control tokens and their placement position. Forcing an anchor that does not match its position into the sequence not only fails to provide effective guidance but may also interfere with the model's original generation capability. In other words, prior research on DLM controllability only answered "can we place it" but did not answer "where to place it and what to place for it to be effective."

Autoregressive models are the foundational architecture of current mainstream large language models, characterized by generating tokens one by one from left to right, with each step depending on all previously generated content. This unidirectional, causal generation approach means control signals (such as system prompts and task instructions) can only be applied at the beginning of the sequence—the so-called "prefix prompt." Prefix fine-tuning injects control capability at the model parameter level but is similarly constrained by the left-to-right processing order. In contrast, diffusion language models employ bidirectional attention, where each position can simultaneously "see" all other positions in the sequence during generation, so control tokens need not be limited to the beginning. However, bidirectional attention also means the generation process is no longer a strict causal chain, and the interaction between control signals and generated content is more complex—this is precisely why AnchorDLM needs to jointly optimize content and position.

AnchorDLM's Core Design: Anchor Generator + Frozen DLM

AnchorDLM's architecture consists of two parts. The first is the anchor generator, which generates an "anchor" for each query—composed of a short output segment and its position in the sequence. The second is the frozen diffusion language model, which receives the anchor as control tokens, initializes the output sequence at the specified position, and guides the subsequent denoising process.

The key to this design is that the DLM's parameters remain unchanged, and all control capability is provided by the external anchor generator. This reduces training costs and avoids potential capability degradation from fine-tuning. The anchor generator is trained using reinforcement learning, with reward signals derived from the answer correctness and constraint satisfaction of the DLM's generated output, allowing the anchor's content and position to be jointly optimized rather than tuned separately.

Reinforcement Learning (RL) is a machine learning paradigm that optimizes policies through interaction with an environment based on reward signals. In AnchorDLM, the anchor generator acts as the "policy," deciding for each query what output segment to generate and where to place it. Since "what constitutes a good anchor" is difficult to directly label with supervised signals, the research adopts reinforcement learning: the anchor generator produces an anchor, the frozen DLM completes denoising based on that anchor and outputs a result, and the system calculates a reward based on the output's answer correctness and constraint satisfaction, then uses that reward to update the anchor generator's parameters. This "generate-evaluate-update" loop enables the anchor's content and position to be jointly optimized around the final task objective, without the need for manually designed position rules or separate parameter tuning.

Explaining the Role of Anchors via Discrete Flow Matching

The paper provides a theoretical explanation from the perspective of Discrete Flow Matching (DFM). Under the DFM framework, the generation process can be viewed as a probability flow guided by a velocity field. The placement of anchors is understood as a reshaping of the initial velocity field—control tokens participating in denoising at specific positions is equivalent to changing the initial conditions of the flow field, making the generation trajectory more inclined toward directions that satisfy task constraints.

This explanation elevates "why anchors work" from empirical observation to a mechanistic level: position is not a hyperparameter but part of the flow field's initial conditions, and the mismatch between content and position is essentially improper initial condition setting.

Discrete Flow Matching (DFM) is a generalization of flow matching methods to discrete data (such as text token sequences). The core idea of flow matching is: rather than directly learning a mapping from noise to data, learn a time-varying velocity field that defines how probability mass "flows" from an initial distribution to a target distribution. In continuous space, this is typically described by ordinary differential equations; in discrete space (such as the probability simplex over a vocabulary), it is modeled through continuous-time Markov chains or similar probability flows. The forward process of diffusion language models gradually adds noise to text, and the reverse process learns a velocity field to reverse this process. AnchorDLM treats anchors as interventions on the initial velocity field of the reverse process: inserting control tokens at specific positions is equivalent to changing the initial velocity of the flow field at that position, thereby guiding the entire generation trajectory toward directions that satisfy constraints.

Experimental Results: Dual Improvements in Controllability and Reasoning Across Five DLM Backbones

The paper conducted experiments on two types of tasks: code generation and mathematical reasoning, covering five DLM backbone models. Results show that AnchorDLM not only improves controllability (i.e., the ability to satisfy task constraints) but also brings additional gains in reasoning performance on multiple benchmarks.

The reasoning improvement is noteworthy. Typically, control mechanisms trade off the model's original capabilities in exchange for constraint satisfaction, but AnchorDLM's results show that both can improve in the same direction. One possible explanation is that reasonable anchors provide structured intermediate hints for the reasoning process, helping the model lock onto the correct reasoning path earlier during denoising.

Backbone model refers to the underlying neural network used as the base architecture, which can load different training weights or adapt to different tasks. In AnchorDLM's experiments, the five DLM backbones may include diffusion language models of different scales or pre-training strategies, such as Transformer-based discrete diffusion models or masked diffusion models. Choosing multiple backbones rather than a single model is to verify whether AnchorDLM's anchor generator has architecture-agnostic generality—that is, regardless of the specific implementation of the underlying DLM, as long as it supports bidirectional attention and iterative denoising, the anchor mechanism will work. The experiments simultaneously cover code generation and mathematical reasoning tasks; the former emphasizes syntax and logical constraints, while the latter emphasizes the correctness of multi-step reasoning. This cross-task design further tests the method's generalization ability.

Implications for Controllable Generation Research

AnchorDLM's joint optimization approach points to a more general issue: in bidirectional attention architectures, control signal design cannot focus solely on content—position itself carries information. This has reference value for any scenario requiring constraints during generation—code completion, structured output, multi-step reasoning.

From an engineering perspective, the approach of freezing the main model and training only a lightweight anchor generator also provides a cost-controllable path for practical deployment. Future directions worth watching include: whether the anchor generator can generalize to constraint types unseen during training, and how anchor length and position granularity affect guidance strength.

One-line Summary

AnchorDLM jointly optimizes anchor content and position to simultaneously improve controllability and reasoning on frozen diffusion language models.