Unverified50% confidenceFactExact time
The researchers computed the principal components of the adapter update matrices and tracked their dynamics as λ varied; when λ causes the teacher to stop thinking, certain directions in the update subspace shift significantly, and this shift point corresponds to the disappearance of the teacher's thinking behavior.
1
Sources
50%
Confidence
Long-term
Relevance
11/4/2026
First Seen
Sources
Related Entities
Related Claims
Unverified自回归生成系统训练采用Teacher Forcing策略，推理阶段切换为自回归模式后产生分布偏移导致误差累积放大74% similarUnverified知识蒸馏的中间层对齐策略（如PKD、TinyBERT）通过强制学生模型的隐藏表示模仿教师模型的分布间接约束嵌入几何结构71% similarUnverifiedA common premise of both knowledge distillation and compiled approaches is that the teacher model's behavior on a specific task has inductively learnable regularity; otherwise, compression or solidification will bring noticeable accuracy loss.70% similarVerifiedThis experiment adopts trajectory-level distillation, which belongs to the category of behavioral cloning, directly having the student model imitate the external behavioral sequences of the teacher model.68% similarUnverifiedIn knowledge distillation, off-policy distillation uses trajectories generated by the teacher model as training samples, and suffers from the distribution shift problem where the state distribution visited by the teacher is inconsistent with the student's own state distribution.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1134024API
curl https://kongchang.com/api/v1/knowledge/claims/1134024MCP
get_claim(id=1134024)