Unverified50% confidenceFactExact time
In soft-label distillation the teacher output carries confidence information, so the student can perceive uncertainty; whereas text-level distillation treats teacher output as deterministic ground truth, so the student inherits the teacher's systematic errors without reservation.
1
Sources
50%
Confidence
Long-term
Relevance
10/17/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTraditional knowledge distillation typically uses soft labels output by the teacher model to transfer dark knowledge such as inter-class similarities.78% similarUnverifiedTraditional distillation uses the teacher's output soft labels (i.e., temperature-scaled class probability distributions) as the supervision signal, whereas current mainstream large language model distillation mostly adopts sequence-level supervision in which the teacher generates responses and the student imitates them.77% similarUnverifiedA white-box teacher allows access to weights and logits, supporting soft-label distillation; a black-box teacher can only provide text outputs, allowing only hard-label offline distillation.73% similarUnverifiedThe teacher-student architecture is a classic paradigm in semi-supervised learning, in which the teacher model generates pseudo-labels for unlabeled samples and the student model is trained using those pseudo-labels as supervision signals.72% similarUnverifiedSoft labels are the probability distribution output by the teacher model after softmax normalization, carrying similarity information between classes, and their information content far exceeds that of a single hard label.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1053605API
curl https://kongchang.com/api/v1/knowledge/claims/1053605MCP
get_claim(id=1053605)