ORL
A training acceleration framework for diffusion language models that, through on-policy residual learning and adjacent entropy contraction, enables the model to determine more tokens in parallel within fewer forward passes, improving inference throughput.
Timeline (last 90 days)
ORL (On-Policy Residual Learning) is an open-source project released by the Reddit machine learning community, aimed at accelerating the inference of diffusion language models (dLLM), covering model weights, training code, evaluation pipeline and inference engine.
ORL adopts on-policy residual learning: training data is generated in real time by the current model to ensure that the training distribution matches the inference distribution.
ORL introduces Adjacent Entropy Contraction, which requires each denoising step to reduce uncertainty relative to the previous step, and gates high-confidence predictions.
According to the author's report in the Reddit post, ORL increases the number of tokens per forward pass of QwenDLM-4B by up to 6.2 times, improves end-to-end throughput by about 6.9 times, and raises HumanEval pass@1 from 82.9% to 89.6%.
On LLaDA-2.1-mini, ORL requires only 4.2M self-generated training tokens and 36 H100 GPU hours of post-training to obtain up to a 3x throughput improvement with comparable quality.
A single RTX 5090 with LLaDA-2.1-mini + ORL reaches 2,788 tokens/s at NVFP4 precision (single-request decoding, batch size 1).
On 8×B200, DiffusionGemma 26B MoE + ORL reaches 7,708 tokens/s at NVFP4 precision (single-request decoding).
On a single H100 and at similar measured accuracy, QwenDLM-4B + ORL reaches 1,298 tokens/s, DFlash 746 tokens/s, and EAGLE3 405 tokens/s (end-to-end system comparison: ORL runs on dlmrt, speculative decoding runs on vLLM).
On the training side, ORL is accompanied by residual state capture, dedicated attention kernels and compute-balanced packing, speeding up training steps by up to 5.2 times.
The entry points provided by the ORL project include the arXiv paper (2610.20687), the project homepage and interactive demo, the training and evaluation code on GitHub, the dlmrt inference engine repository, and the checkpoints on Hugging Face.
All Facts (10)
ORL (On-Policy Residual Learning) is an open-source project released by the Reddit machine learning community, aimed at accelerating the inference of diffusion language models (dLLM), covering model weights, training code, evaluation pipeline and inference engine.
50%UnverifiedORL adopts on-policy residual learning: training data is generated in real time by the current model to ensure that the training distribution matches the inference distribution.
50%UnverifiedORL introduces Adjacent Entropy Contraction, which requires each denoising step to reduce uncertainty relative to the previous step, and gates high-confidence predictions.
50%UnverifiedAccording to the author's report in the Reddit post, ORL increases the number of tokens per forward pass of QwenDLM-4B by up to 6.2 times, improves end-to-end throughput by about 6.9 times, and raises HumanEval pass@1 from 82.9% to 89.6%.
50%UnverifiedOn LLaDA-2.1-mini, ORL requires only 4.2M self-generated training tokens and 36 H100 GPU hours of post-training to obtain up to a 3x throughput improvement with comparable quality.
50%UnverifiedA single RTX 5090 with LLaDA-2.1-mini + ORL reaches 2,788 tokens/s at NVFP4 precision (single-request decoding, batch size 1).
50%UnverifiedOn 8×B200, DiffusionGemma 26B MoE + ORL reaches 7,708 tokens/s at NVFP4 precision (single-request decoding).
50%UnverifiedOn a single H100 and at similar measured accuracy, QwenDLM-4B + ORL reaches 1,298 tokens/s, DFlash 746 tokens/s, and EAGLE3 405 tokens/s (end-to-end system comparison: ORL runs on dlmrt, speculative decoding runs on vLLM).
50%UnverifiedOn the training side, ORL is accompanied by residual state capture, dedicated attention kernels and compute-balanced packing, speeding up training steps by up to 5.2 times.
50%UnverifiedThe entry points provided by the ORL project include the arXiv paper (2610.20687), the project homepage and interactive demo, the training and evaluation code on GitHub, the dlmrt inference engine repository, and the checkpoints on Hugging Face.
50%