Unverified50% confidenceFactExact time
In knowledge distillation, the smaller the student model, the more pronounced the accuracy drop.
1
Sources
50%
Confidence
Long-term
Relevance
11/6/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedKnowledge distillation has the student network learn the soft probability distribution output by the teacher network, enabling the student model to retain core reasoning capabilities while reducing its parameter count by more than 10 times.77% similarUnverified实验数据显示,经过5-10轮迭代训练后,模型在低频词汇和长尾知识上的覆盖率下降超过40%,而输出文本在表面流畅性上几乎没有可察觉的退化76% similarUnverifiedWhen the teacher model grows larger while the critic remains small and fixed, the quality of the student model instead degrades, and training collapse even occurs.76% similarUnverifiedResearch has found that information asymmetry is one of the important reasons for the lower enrollment and completion rates of first-generation college students.73% similarVerifiedAnthropic's research shows that even mild alignment training can cause a 2–8 percentage point decline in model performance on certain reasoning-intensive tasks.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1141680API
curl https://kongchang.com/api/v1/knowledge/claims/1141680MCP
get_claim(id=1141680)