Policy Distillation
A general paradigm for transferring the behavior of a teacher model to a student model by minimizing the divergence (such as KL divergence) between the student's output distribution and the teacher's output distribution.
Timeline (last 90 days)
The endpoint-style policy distillation method uses the difference in log probabilities of the teacher model before and after RL training as the supervision signal, applied to trajectories generated by the student model itself.
In the RL post-training scenario, the teacher model's output distribution keeps changing as training proceeds, and the distillation supervision signal becomes 'the relative change in the teacher policy' rather than the absolute output.
The paper distilled the learned manipulation policies into RGB-based policies and completed zero-shot transfer of several challenging assembly tasks on real hardware.
Policy distillation typically refers to using a student policy to imitate the behavior of one or more teacher policies.
All Facts (4)
The endpoint-style policy distillation method uses the difference in log probabilities of the teacher model before and after RL training as the supervision signal, applied to trajectories generated by the student model itself.
50%UnverifiedIn the RL post-training scenario, the teacher model's output distribution keeps changing as training proceeds, and the distillation supervision signal becomes 'the relative change in the teacher policy' rather than the absolute output.
50%UnverifiedThe paper distilled the learned manipulation policies into RGB-based policies and completed zero-shot transfer of several challenging assembly tasks on real hardware.
50%UnverifiedPolicy distillation typically refers to using a student policy to imitate the behavior of one or more teacher policies.
50%