Unverified50% confidenceFactExact time
In knowledge distillation, off-policy distillation uses trajectories generated by the teacher model as training samples, and suffers from the distribution shift problem where the state distribution visited by the teacher is inconsistent with the student's own state distribution.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn on-policy distillation, the student model first generates trajectories itself, and then the teacher model provides supervision signals for these paths, thereby avoiding the distribution shift between offline data and student behavior.78% similarUnverifiedKnowledge distillation causes student models to inherit certain behavioral patterns of teacher models, resulting in false positives from fingerprint cross-contamination during attribution analysis.77% similarUnverified知识蒸馏的中间层对齐策略（如PKD、TinyBERT）通过强制学生模型的隐藏表示模仿教师模型的分布间接约束嵌入几何结构77% similarUnverifiedOn-policy self-distillation computes the distillation loss on trajectories sampled by the student itself, conditioning on the states produced by the student's current policy to let the teacher provide a better target distribution, thereby reducing error accumulation caused by distribution shift.77% similarUnverified自回归生成系统训练采用Teacher Forcing策略，推理阶段切换为自回归模式后产生分布偏移导致误差累积放大75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1046838API
curl https://kongchang.com/api/v1/knowledge/claims/1046838MCP
get_claim(id=1046838)