Unverified50% confidenceFactExact time
Forward KL tends to make the student policy cover all modes of the teacher policy, learning comprehensively but possibly not precisely enough; reverse KL tends to make the student policy concentrate on the high-probability regions of the teacher policy, learning precisely but possibly missing certain modes.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
Unverified知识蒸馏的中间层对齐策略（如PKD、TinyBERT）通过强制学生模型的隐藏表示模仿教师模型的分布间接约束嵌入几何结构76% similarUnverifiedThe challenge of streaming model distillation lies in the fact that if the teacher model adopts a non-causal architecture, its latent representations contain future information that the student model cannot access during inference, requiring the design of alignment strategies to prevent the student from being forced to fit information it cannot reproduce.72% similarUnverifiedPolicy distillation typically refers to using a student policy to imitate the behavior of one or more teacher policies.69% similarUnverifiedLISCO (Learned Iterative Solver for Constrained Optimization) learns to predict primal-dual step sizes to iteratively approach a solution satisfying KKT optimality conditions, rather than directly predicting the final control output.69% similarUnverified自回归生成系统训练采用Teacher Forcing策略，推理阶段切换为自回归模式后产生分布偏移导致误差累积放大68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1023683API
curl https://kongchang.com/api/v1/knowledge/claims/1023683MCP
get_claim(id=1023683)