Unverified50% confidenceFactExact time
Distillation can be decomposed into four dimensions: hard labels/soft labels, data source, offline/online timing, and teacher identity.
1
Sources
50%
Confidence
Long-term
Relevance
9/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedA white-box teacher allows access to weights and logits, supporting soft-label distillation; a black-box teacher can only provide text outputs, allowing only hard-label offline distillation.66% similarUnverified跨分词器蒸馏的标准做法是将教师top-K logits投影到学生token上，在分段边界处把教师token映射到对应学生token序列的第一个token，这是一种多对一映射63% similarUnverified在条件生成中，每个标签通常被映射为固定维度（通常64-512维）的可学习嵌入向量，多个标签的嵌入向量通过求和、拼接或注意力池化等方式聚合为单一条件向量59% similarUnverifiedTraditional knowledge distillation typically uses soft labels output by the teacher model to transfer dark knowledge such as inter-class similarities.59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/911347API
curl https://kongchang.com/api/v1/knowledge/claims/911347MCP
get_claim(id=911347)