Student Model

In knowledge distillation, the target model trained to match the teacher model's distribution, optimized via forward KL divergence so as to cover all modes of the teacher distribution.