Distillation, quantization, and pruning are often collectively called the three-piece model compression toolkit, but distillation changes the training signal, while the latter two directly compress existing weights.