Verified65% confidenceFactExact time
Hard distillation refers to the student model directly learning the final output labels or generated text of the teacher model, while soft distillation requires the student model to learn the complete probability distribution (logits) output by the teacher model.
3
Sources
65%
Confidence
Long-term
Relevance
8/7/2026
First Seen
Sources
Related Claims
VerifiedModel distillation softens the teacher model's softmax outputs via temperature scaling, allowing information from low-probability classes to be transferred to the student model.79% similarUnverifiedDistillation can be viewed as an implicit, data-constrained form of model merging—distilling a student using the averaged outputs of multiple teacher models is effectively performing a soft merge at the level of output distributions.78% similarUnverifiedOutput distillation involves directly using text generated by a teacher model—such as dialogues and reasoning chains—as training data to fine-tune a student model, without requiring access to the teacher's internal parameters.75% similarVerified模型蒸馏是一种将大型教师模型能力迁移到更小学生模型的技术，可能被用于扩散危险能力74% similarUnverifiedIn traditional white-box distillation, the student model can directly access the complete probability distribution (logits) of the teacher model's output layer, whereas commercial large language model APIs typically return only the final generated text or top-k token probabilities.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/704190API
curl https://kongchang.com/api/v1/knowledge/claims/704190MCP
get_claim(id=704190)