Unverified50% confidenceFactExact time
Soft labels are the probability distribution output by the teacher model after softmax normalization, carrying similarity information between classes, and their information content far exceeds that of a single hard label.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTraditional knowledge distillation typically uses soft labels output by the teacher model to transfer dark knowledge such as inter-class similarities.70% similarUnverifiedDistillation can be decomposed into four dimensions: hard labels/soft labels, data source, offline/online timing, and teacher identity.64% similarUnverifiedA white-box teacher allows access to weights and logits, supporting soft-label distillation; a black-box teacher can only provide text outputs, allowing only hard-label offline distillation.59% similarUnverifiedSoft compression operates in vector space, encoding entire text passages into a small number of dense continuous embedding vectors that can be passed as soft prompts into the attention layers of a language model.58% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1013962API
curl https://kongchang.com/api/v1/knowledge/claims/1013962MCP
get_claim(id=1013962)