Verified70% confidenceFactExact time
Model distillation technology was proposed by Geoffrey Hinton's team in 2015, originally used for model compression, by having a student model fit the soft-label outputs of a teacher model.
4
Sources
70%
Confidence
Long-term
Relevance
7/17/2026
First Seen
Sources
Related Claims
Verified模型蒸馏（Knowledge Distillation）最初由 Hinton 等人于2015年提出，核心思想是让小模型学习大模型的软标签而非硬标签78% similarVerifiedModel distillation was first proposed by Hinton et al. in 2015; by having a small model learn the output distribution of a large model, the student model can often retain 80% to 95% of the teacher model's performance.77% similarVerifiedModel distillation softens the teacher model's softmax outputs via temperature scaling, allowing information from low-probability classes to be transferred to the student model.75% similarVerifiedHard distillation refers to the student model directly learning the final output labels or generated text of the teacher model, while soft distillation requires the student model to learn the complete probability distribution (logits) output by the teacher model.73% similarVerifiedModel Distillation in machine learning refers to training a small student model using the outputs of a large teacher model, in order to compress model size while retaining capability.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/538841API
curl https://kongchang.com/api/v1/knowledge/claims/538841MCP
get_claim(id=538841)