[KongchangAI]
ConceptOff-policy Distillation / offline蒸馏

Off-Policy Distillation

A model distillation approach in which training data is generated offline by the teacher model before training begins. The student model learns from a fixed dataset. This approach is contrasted with on-policy distillation.

Timeline (last 90 days)

Nov 10

Off-policy distillation directly uses trajectories generated by the teacher; the student is exposed to the teacher's distribution, which deviates from its own test-time behavior and can easily cause forgetting or degraded generalization.

Unverified50%
Nov 10

On held-out test sets, SSD consistently outperforms off-policy distillation.

Unverified50%

All Facts (2)

Source Articles