Conceptsimilarity distribution matching
Similarity Distribution Matching
The core operation of SimOPD, in which a KL divergence or cross-entropy loss drives the similarity distribution in the student representation space toward that of the teacher representation space. It constrains only the relative distance structure and does not require point-by-point feature alignment.
Timeline (last 90 days)
Oct 19
The core operation of SimOPD is to make the similarity distribution induced by the response representations output by the student model approach the similarity distribution induced by the teacher model's internal representations, using KL divergence or cross-entropy loss to bring the two distributions closer.
Unverified50%