Unverified50% confidenceFactExact time
When the teacher model grows larger while the critic remains small and fixed, the quality of the student model instead degrades, and training collapse even occurs.
1
Sources
50%
Confidence
Long-term
Relevance
11/2/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe cost of distillation lies in the fact that the student model tends to inherit the teacher model's behavior within the training distribution, but its generalization ability on out-of-distribution difficult samples (rare words, strong noise, rare accents) is markedly weaker than the original large model.71% similarUnverifiedThrough distillation, student models with only one-tenth the parameters of the teacher model can often retain 80%–95% of the teacher model's performance.69% similarUnverified牛津大学2023年的一项研究量化了模型坍缩效应,仅经过五代自我训练,文本多样性指标下降了约40%68% similarVerifiedA 2023 joint study by Oxford University, Cambridge University, and other institutions elaborated on the model collapse effect: when a model is iteratively trained on its own generated data, the statistical variance of outputs continuously decreases, rare long-tail knowledge disappears first, and the model ultimately converges to a cultural mean state.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1125599API
curl https://kongchang.com/api/v1/knowledge/claims/1125599MCP
get_claim(id=1125599)