Unverified50% confidenceFactExact time
SDPO's direct-matching self-teacher distribution policy may destroy existing successful behaviors on correct rollouts.
1
Sources
50%
Confidence
Long-term
Relevance
10/20/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFor incorrect rollouts, APSD adopts an extrapolation strategy, extending the target distribution beyond the self-teacher to strengthen correction.74% similarUnverifiedSMD also adds a repulsion mechanism: keeping the student away from the teacher's weaker predictions, to avoid inheriting the teacher's unreliable prediction patterns at certain time steps.69% similarUnverifiedReverse KL divergence tends to make the student focus on a few high-probability modes of the teacher (mode-seeking), and when the teacher has a sharper, more trustworthy distribution because it carries the Reminder, it is more suitable as the correction target for failed rollouts.68% similarUnverifiedOff-policy reinforcement learning reuses samples generated by older policies, which brings efficiency advantages but introduces distribution shift and high-variance gradient estimation problems.67% similarVerifiedBehavioral cloning simplifies the problem to supervised learning, but suffers from distribution shift, where differences between the state distributions at training time and execution time lead to error accumulation.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1074083API
curl https://kongchang.com/api/v1/knowledge/claims/1074083MCP
get_claim(id=1074083)