Unverified50% confidenceFactExact time
In on-policy distillation, the student model generates its own rankings during training and the teacher provides feedback on these rankings, thereby avoiding the mismatch between teacher and student distributions (exposure bias) in offline distillation.
1
Sources
50%
Confidence
Long-term
Relevance
11/2/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedOnline policy distillation (OPD) has the student model generate its own training responses, which are then supervised by the teacher model.79% similarUnverifiedToken-level on-policy distillation aligns the student model's hidden states with teacher signals and adds a gating mechanism for credit assignment, yielding improvements on math and tool-use benchmarks.79% similarUnverifiedThe training data for on-policy distillation is generated by the interaction of the student's current policy with the environment, and the state distribution changes as the student policy is updated, in line with the DAgger idea, which can mitigate distribution shift.78% similarUnverified教师-学生蒸馏框架先训练可访问完整环境信息的教师策略，再蒸馏到仅依赖机载传感器的学生策略中以实现真实硬件部署76% similarUnverifiedDINOv2的核心训练策略基于自蒸馏，模型同时维护教师网络和学生网络，学生网络通过预测教师网络对同一图像不同裁剪视角的输出来学习特征表示76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1125725API
curl https://kongchang.com/api/v1/knowledge/claims/1125725MCP
get_claim(id=1125725)