Unverified50% confidenceFactExact time
Offline distillation trains the student using a static dataset pre-generated by the teacher model, which leads to the problem of covariate shift.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWhen model parameters and data volume reach a certain scale, some capabilities not explicitly taught during training emerge, such as few-shot learning, chain-of-thought reasoning, and cross-lingual transfer.73% similarUnverifiedThe paper observes that supervision signals from a fixed teacher become increasingly less worthwhile as the student model's capabilities improve71% similarUnverified训练数据中的混杂变量是导致模型学到捷径学习（shortcut learning）的重要原因71% similarUnverified训练指标的周期性波动往往与训练数据的遍历方式有关，若每个epoch数据顺序固定或困难样本固定位置出现会导致规律性起伏70% similarUnverifiedWhen the distribution of online data deviates from the training data, model performance degrades (data drift and model decay), requiring retraining to be triggered.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1001132API
curl https://kongchang.com/api/v1/knowledge/claims/1001132MCP
get_claim(id=1001132)