Unverified50% confidenceFactExact time
The researchers found that even starting from the states where the student deviates the most, the teacher's own takeover can still complete the task, whereas the student, after training on the teacher's per-token scores, often runs out of turns.
1
Sources
50%
Confidence
Long-term
Relevance
10/21/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe paper observes that supervision signals from a fixed teacher become increasingly less worthwhile as the student model's capabilities improve73% similarUnverifiedOn-policy distillation requires the student to first generate its own prefix trajectory, after which the teacher provides guidance at the states the student actually reaches, so the state distributions are aligned, and it is usually more stable on long-horizon tasks where errors tend to accumulate.70% similarUnverifiedThe paper concludes formally with the analogy of a wise teacher's intervention, where the teacher can advance a learner to the next level when they stagnate at a plateau70% similarUnverifiedThe limitation of distillation is that the student model's capability ceiling is constrained by the teacher's performance on that task, and it is difficult to generalize to new tasks outside the training distribution.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1078837API
curl https://kongchang.com/api/v1/knowledge/claims/1078837MCP
get_claim(id=1078837)