Knowledge Distillation is a technique for transferring knowledge from a large teacher model to a smaller student model, proposed by Geoffrey Hinton et al. in 2015