Unverified50% confidenceFactExact time
On-policy distillation requires the student to first generate its own prefix trajectory, after which the teacher provides guidance at the states the student actually reaches, so the state distributions are aligned, and it is usually more stable on long-horizon tasks where errors tend to accumulate.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe paper observes that supervision signals from a fixed teacher become increasingly less worthwhile as the student model's capabilities improve72% similarUnverifiedIn self-distillation, if the teacher is an EMA or synchronous variant of the student policy, each update of the student changes the teacher, causing the target signal to drift and turning training into chasing a moving target.71% similarUnverifiedThe paper concludes formally with the analogy of a wise teacher's intervention, where the teacher can advance a learner to the next level when they stagnate at a plateau71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1046839API
curl https://kongchang.com/api/v1/knowledge/claims/1046839MCP
get_claim(id=1046839)