GAD-RL: Improving OCR Transcription Faithfulness with Gated and Attenuated Distillation

The "Overcorrection" Problem in OCR

Vision-language models (VLMs) have a subtle flaw when recognizing text in images: when the image contains abnormal, incomplete, or linguistically unusual text, the model tends to "rewrite" it into something more grammatically natural. This behavior seems reasonable on the surface, but it fundamentally undermines the most critical requirement of OCR transcription — faithfulness. For documents like IDs, receipts, and professional records, the model's unsolicited "corrections" can introduce serious errors.

The GAD-RL method proposed in a new arXiv paper, Improving OCR Faithfulness via Gated and Attenuated On-Policy Distillation, is designed precisely to address this problem. The core idea is to make task rewards and teacher guidance work together during post-training, while also solving the gradual imbalance between the two over the course of training.

Why a Fixed Teacher Eventually "Stops Helping"

One key observation in the paper comes from an offline analysis: the supervisory signal from a fixed teacher becomes increasingly inefficient as the student model improves.

This manifests along two dimensions. First, along the temporal axis of training checkpoints — the student clearly benefits from the teacher early in training, but once the student's own capabilities catch up, maintaining the same level of teacher guidance can actually hurt performance. Second, along the response group dimension — for sample groups that already achieve high task rewards, the marginal value of additional teacher guidance is relatively limited.

In other words, sequence-level task rewards and local teacher guidance are complementary by nature, but fixed-weight distillation cannot adapt to the student's dynamic growth. This provides direct motivation for an adaptive regulation mechanism.

On-Policy Distillation (OPD) differs from traditional offline distillation in that the teacher model scores or guides outputs generated by the student (i.e., the student's "policy samples") rather than from a fixed static dataset. This keeps the teacher signal aligned with the student's current behavioral distribution, making it theoretically more efficient than offline distillation. However, the on-policy setting introduces a new problem: once the student has surpassed or approached the teacher's guidance level on certain samples, forcing the same distillation loss weight will interfere with the reinforcement learning task reward signal — a "knowledge drag" effect. GAD-RL was developed after observing this phenomenon, treating the teacher as an auxiliary signal with dynamic weighting rather than a fixed constraint.

GAD-RL's Three-Part Regulation Mechanism

GAD-RL (Gated and Attenuated Distillation RL) dynamically adjusts the intensity of teacher supervision based on the student's current task performance and local distribution during joint post-training. It consists of three key design components:

Gating: Fully Disable Distillation for High-Reward Groups

A frozen teacher model performs conditional predictions based on reference transcription text and the student's generated prefix. When any output in a response group achieves a task reward of 0.95 or higher, GAD-RL completely disables distillation for that group. The logic is straightforward: the student is already performing well enough on these samples, and teacher intervention would only be redundant or even harmful.

Attenuation: Continuously Weaken as Reward Increases

Beyond hard gating, GAD-RL also applies a continuous attenuation mechanism — as the group-mean reward increases, distillation strength is progressively diminished. This ensures a smooth inverse relationship between supervision intensity and student capability at all times, avoiding the abrupt transitions that gating alone would introduce.

Probability-Weighted Forward KL

At the local level, GAD-RL weights the forward KL divergence by the student's predicted probability for the teacher's Top-1 token. When the student has very low support for the teacher's recommended candidate token, the corresponding local auxiliary update is appropriately suppressed. This design prevents forcibly pulling the student toward a direction it currently "doesn't agree with," making local guidance more robust.

GRPO (Group Relative Policy Optimization) is the reinforcement learning training framework on which GAD-RL is built. Its core idea is to sample a group of responses for the same input and estimate the policy gradient using group-relative rewards rather than absolute rewards, avoiding the need for an accurate value function. Compared to PPO, GRPO does not require a separate value network, reducing computational overhead, and has been widely adopted in post-training of large language models. GAD-RL introduces a teacher distillation component on top of GRPO, so each parameter update is jointly influenced by task reward gradients and teacher KL constraints, with the three-part regulation mechanism dynamically allocating weight between the two.

Experimental Results: Significant Improvements Over Baseline

On the Qwen3.5-2B model, GAD-RL delivers impressive results:

• On CHAOS-Bench, it achieves 59.92% Micro Recall, outperforming GRPO and fixed-weight GRPO+OPD by 8.45 percentage points and 4.43 percentage points, respectively.
• On OmniDocBench v1.6, it achieves an Overall score of 91.18.

These numbers demonstrate a clear advantage of adaptive distillation regulation over fixed-weight strategies. The 4.43 percentage point gain on CHAOS-Bench — a benchmark specifically designed to test handling of anomalous text — directly validates the value of "dynamically reducing teacher guidance": it allows the model to benefit from early teacher guidance while maintaining faithfulness to the original text once it matures.

CHAOS-Bench is a benchmark focused on OCR faithfulness evaluation, designed specifically to include large amounts of "counterintuitive" text: misspellings, abbreviations, number strings, mixed symbols, and other content that language models tend to "correct." Micro Recall in this context measures character-level recall of the original character sequence, directly reflecting how well the transcription preserves the source text rather than its linguistic fluency. High Micro Recall means the model can suppress the urge to rewrite even when faced with content that defies linguistic intuition. The choice of this metric itself highlights the fundamental difference between OCR tasks and general text generation tasks: the latter encourages models to apply their linguistic knowledge, while the former requires models to exercise restraint in the face of it.

Significance and Limitations

The value of GAD-RL lies in how it cleverly transfers the explore-exploit trade-off from reinforcement learning into the context of knowledge distillation: the teacher is no longer an unchanging authority, but a tutor who gradually "steps back" as the student grows. This on-policy dynamic distillation approach also has broader implications for other generative tasks that need to balance task rewards with teacher guidance.

Of course, the paper's validation is currently focused on the OCR faithfulness setting and the Qwen3.5-2B scale. Generalization to larger models and more diverse tasks remains to be tested. But based on the published results, GAD-RL offers an elegantly designed and effective solution to the pain point of VLMs "helpfully" rewriting source text in OCR tasks.