Verified80% confidenceFactExact time
The core idea of knowledge distillation is to have the student model imitate the soft labels (probability distributions across categories) output by the teacher model, with soft labels being referred to by Hinton as "dark knowledge".
7
Sources
80%
Confidence
Long-term
Relevance
7/14/2026
First Seen
Sources
跨分词器知识蒸馏：85%信息丢失的根因与链式法则解法
Related Claims
VerifiedKnowledge distillation, proposed by Hinton et al., has student models inherit implicit knowledge by learning from the soft probability distributions output by teacher models.83% similarUnverified知识蒸馏由Hinton等人2015年提出，通过温度参数软化概率分布，使低概率类别的暗知识更显著供学生模型学习82% similarVerified知识蒸馏由Hinton等人在2015年提出，核心思想是让小型学生模型模仿大型教师模型的输出分布77% similarVerifiedKnowledge distillation was first proposed by Hinton, Vinyals, and Dean at Google in 2015, with the core idea of having a small student model learn from the soft labels output by a large teacher model.76% similarUnverifiedModel distillation is a technique in deep learning for transferring knowledge from a teacher model to a student model.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/508975API
curl https://kongchang.com/api/v1/knowledge/claims/508975MCP
get_claim(id=508975)