Unverified90% confidenceFactExact time
The core idea of knowledge distillation is to have the small model learn the probability distribution (soft labels) of the large model's outputs rather than just hard labels
1
Sources
90%
Confidence
Long-term
Relevance
8/2/2026
First Seen
Sources
Related Entities
Related Claims
Unverified模型蒸馏（Knowledge Distillation）是将大型模型的知识迁移至小型轻量模型的技术，可在参数量缩减数十倍的同时保留大部分性能82% similarUnverified知识蒸馏（Knowledge Distillation）由Hinton等人在2015年系统提出，核心思想是用大模型的输出概率分布指导小模型训练81% similarVerified模型蒸馏（Knowledge Distillation）最初由 Hinton 等人于2015年提出，核心思想是让小模型学习大模型的软标签而非硬标签78% similarUnverified知识蒸馏通过软标签传递教师模型输出的完整概率分布信息，起到隐式数据增强效果74% similarUnverified轻量模型通过知识蒸馏和模型剪枝技术压缩规模，知识蒸馏让小模型模仿大模型的输出概率分布，剪枝移除对输出贡献最小的网络权重72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/679830API
curl https://kongchang.com/api/v1/knowledge/claims/679830MCP
get_claim(id=679830)