Online Distillation
A knowledge distillation training paradigm in which a student model receives supervision signals from a teacher model on token sequences generated by its own policy, aiming to eliminate the distribution shift between training and inference.
Timeline (last 90 days)
The fundamental blind spot of online distillation is that states the teacher can reach but the student rarely or never reaches receive almost no supervision signal, because the student's extremely low sampling probability leaves these regions unable to obtain gradients.
The paper "CoRD: Coupled Rollouts Fill the Blind Spot of On-Policy Distillation" proposes the coupled rollout distillation (CoRD) method, aimed at actively pushing the distribution of training states toward the teacher side without abandoning the online loss.
On five public math benchmarks, compared with the same training steps and data volume, CoRD improves average pass@1 by 0.9 to 2.0 percentage points and pass@8 by 1.2 to 3.4 percentage points over online distillation.
In experiments with a frozen teacher-reachable state, the gap reduction achieved by the 4B student trained with CoRD within 16 steps was still not reached by online distillation after 80 steps.
All Facts (4)
The fundamental blind spot of online distillation is that states the teacher can reach but the student rarely or never reaches receive almost no supervision signal, because the student's extremely low sampling probability leaves these regions unable to obtain gradients.
50%UnverifiedThe paper "CoRD: Coupled Rollouts Fill the Blind Spot of On-Policy Distillation" proposes the coupled rollout distillation (CoRD) method, aimed at actively pushing the distribution of training states toward the teacher side without abandoning the online loss.
50%UnverifiedOn five public math benchmarks, compared with the same training steps and data volume, CoRD improves average pass@1 by 0.9 to 2.0 percentage points and pass@8 by 1.2 to 3.4 percentage points over online distillation.
50%UnverifiedIn experiments with a frozen teacher-reachable state, the gap reduction achieved by the 4B student trained with CoRD within 16 steps was still not reached by online distillation after 80 steps.
50%