Verified75% confidenceFactExact time
Knowledge distillation was first proposed by Hinton, Vinyals, and Dean at Google in 2015, with the core idea of having a small student model learn from the soft labels output by a large teacher model.
3
Sources
75%
Confidence
Long-term
Relevance
7/22/2026
First Seen
Sources
Qwen4、DeepSeek V4、GLM同台竞技：国产大模型密集冲刺
Related Claims
Verified知识蒸馏由Hinton等人在2015年提出，核心思想是让小型学生模型模仿大型教师模型的输出分布84% similarVerifiedThe core idea of knowledge distillation is to have the student model imitate the soft labels (probability distributions across categories) output by the teacher model, with soft labels being referred to by Hinton as "dark knowledge".76% similarVerifiedKnowledge distillation, proposed by Hinton et al., has student models inherit implicit knowledge by learning from the soft probability distributions output by teacher models.75% similarUnverified知识蒸馏最早由Hinton等人在2015年提出，DiffusionGemma创造性地将其用于步数压缩，教师模型用较多步数（如48步）生成，学生模型用更少步数（如12步）达到相近质量70% similarVerifiedKnowledge distillation was formally proposed by Hinton, Vinyals, and Dean in 2015 in the paper "Distilling the Knowledge in a Neural Network"69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/593851API
curl https://kongchang.com/api/v1/knowledge/claims/593851MCP
get_claim(id=593851)