AnchorDLM
A controllable generation framework for diffusion language models, consisting of an anchor generator and a frozen diffusion language model. Reinforcement learning jointly optimizes the content and placement of control tokens, and the reshaping effect of anchors on the initial velocity field is interpreted from the perspective of discrete flow matching.
Timeline (last 90 days)
The AnchorDLM framework consists of two parts: an anchor generator and a frozen diffusion language model.
The anchor generator of AnchorDLM generates for each query an anchor composed jointly of a short output segment and its position in the sequence.
In AnchorDLM, the main diffusion language model parameters are not modified, and the control capability is provided entirely by the external anchor generator, in order to reduce training cost and avoid capability degradation caused by fine-tuning.
The anchor generator of AnchorDLM is trained using reinforcement learning, with the reward signal coming from the answer correctness and degree of constraint satisfaction of the DLM-generated output.
The paper explains the role of anchors from the perspective of discrete flow matching (DFM): anchor placement is understood as a reshaping of the initial velocity field, and the position is part of the initial conditions of the flow field.
The paper conducts experiments on two types of tasks, code generation and mathematical reasoning, covering five DLM backbone models.
Experimental results show that AnchorDLM not only improves controllability but also brings additional gains in reasoning performance on multiple benchmarks.
The anchor mechanism in AnchorDLM may help the model lock onto the correct reasoning path earlier by providing structured intermediate hints for the reasoning process.
The approach of freezing the main model and training only a lightweight anchor generator provides a cost-controllable path for practical deployment.
All Facts (9)
The AnchorDLM framework consists of two parts: an anchor generator and a frozen diffusion language model.
50%UnverifiedThe anchor generator of AnchorDLM generates for each query an anchor composed jointly of a short output segment and its position in the sequence.
50%UnverifiedIn AnchorDLM, the main diffusion language model parameters are not modified, and the control capability is provided entirely by the external anchor generator, in order to reduce training cost and avoid capability degradation caused by fine-tuning.
50%UnverifiedThe anchor generator of AnchorDLM is trained using reinforcement learning, with the reward signal coming from the answer correctness and degree of constraint satisfaction of the DLM-generated output.
50%UnverifiedThe paper explains the role of anchors from the perspective of discrete flow matching (DFM): anchor placement is understood as a reshaping of the initial velocity field, and the position is part of the initial conditions of the flow field.
50%UnverifiedThe paper conducts experiments on two types of tasks, code generation and mathematical reasoning, covering five DLM backbone models.
50%UnverifiedExperimental results show that AnchorDLM not only improves controllability but also brings additional gains in reasoning performance on multiple benchmarks.
50%UnverifiedThe anchor mechanism in AnchorDLM may help the model lock onto the correct reasoning path earlier by providing structured intermediate hints for the reasoning process.
50%UnverifiedThe approach of freezing the main model and training only a lightweight anchor generator provides a cost-controllable path for practical deployment.
50%