Unverified50% confidenceFactExact time
Traditional distillation uses the teacher's output soft labels (i.e., temperature-scaled class probability distributions) as the supervision signal, whereas current mainstream large language model distillation mostly adopts sequence-level supervision in which the teacher generates responses and the student imitates them.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSoft labels are the probability distribution output by the teacher model after softmax normalization, carrying similarity information between classes, and their information content far exceeds that of a single hard label.75% similarUnverifiedTraditional knowledge distillation typically uses soft labels output by the teacher model to transfer dark knowledge such as inter-class similarities.69% similarUnverifiedA white-box teacher allows access to weights and logits, supporting soft-label distillation; a black-box teacher can only provide text outputs, allowing only hard-label offline distillation.68% similarUnverifiedCommon directions for improving pseudo-label quality include confidence filtering, introducing language model rescoring (LM rescoring), and ensembling multiple teacher models.67% similarUnverifiedThe teacher-student architecture is a classic paradigm in semi-supervised learning, in which the teacher model generates pseudo-labels for unlabeled samples and the student model is trained using those pseudo-labels as supervision signals.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1023605API
curl https://kongchang.com/api/v1/knowledge/claims/1023605MCP
get_claim(id=1023605)