Unverified50% confidenceSolutionExact time
The core operation of SimOPD is to make the similarity distribution induced by the response representations output by the student model approach the similarity distribution induced by the teacher model's internal representations, using KL divergence or cross-entropy loss to bring the two distributions closer.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
Unverified对比学习通过最大化同类样本相似度、最小化异类样本相似度来学习表征，SimCLR、MoCo等方法均基于此原理73% similarUnverified知识蒸馏的中间层对齐策略（如PKD、TinyBERT）通过强制学生模型的隐藏表示模仿教师模型的分布间接约束嵌入几何结构68% similarUnverifiedThe paper uses random matrix theory to derive deterministic equivalents of the population error for an optimally trained teacher model and a student model trained via gradient flow67% similarUnverifiedExperiments show that LoopOPD significantly improves the model's mathematical reasoning ability, and D-LoopOPD brings further gains through dynamic teacher updates.67% similarUnverifiedOn-policy self-distillation computes the distillation loss on trajectories sampled by the student itself, conditioning on the states produced by the student's current policy to let the teacher provide a better target distribution, thereby reducing error accumulation caused by distribution shift.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1069445API
curl https://kongchang.com/api/v1/knowledge/claims/1069445MCP
get_claim(id=1069445)