Unverified50% confidenceFactExact time
Reverse KL divergence tends to make the student focus on a few high-probability modes of the teacher (mode-seeking), and when the teacher has a sharper, more trustworthy distribution because it carries the Reminder, it is more suitable as the correction target for failed rollouts.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSMD also adds a repulsion mechanism: keeping the student away from the teacher's weaker predictions, to avoid inheriting the teacher's unreliable prediction patterns at certain time steps.74% similarUnverifiedWhen the teacher has non-uniform style preferences, SFT produces off-axis style drift along a non-zero task mean under population updates, which becomes the root cause of forgetting.71% similarUnverifiedIn structure learning tasks, a carefully designed estimation bias with a controllable direction is sometimes more useful than pursuing unbiased or uniformly small errors.67% similarUnverifiedAny study comparing fine-tuning schemes with different numbers of trainable parameters should be wary of the shared learning rate trap; otherwise, conclusions that fewer parameters yield better results may need to be revisited.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1046845API
curl https://kongchang.com/api/v1/knowledge/claims/1046845MCP
get_claim(id=1046845)