Latent Space Distillation

A variant of knowledge distillation that shifts the alignment point from the output layer to the intermediate hidden-layer feature representation space. The student model learns the teacher model's intermediate representations rather than its final outputs, making it suitable for encoder-type models whose outputs are continuous vectors rather than classification probabilities.