Unverified50% confidenceFactExact time
For incorrect rollouts, APSD adopts an extrapolation strategy, extending the target distribution beyond the self-teacher to strengthen correction.
1
Sources
50%
Confidence
Long-term
Relevance
10/20/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSDPO's direct-matching self-teacher distribution policy may destroy existing successful behaviors on correct rollouts.74% similarUnverifiedThe researchers propose APSD (Asymmetric Policy Optimization via Self-Distillation), which balances behavior preservation and error correction by constructing a correctness-dependent asymmetric target distribution.72% similarUnverifiedSMD also adds a repulsion mechanism: keeping the student away from the teacher's weaker predictions, to avoid inheriting the teacher's unreliable prediction patterns at certain time steps.69% similarUnverifiedUnder off-policy sampling conditions, when the preference margin is aligned in direction with the utility gradient, DPO's update shifts from a corrective signal to a reinforcement-style update similar to reinforcement learning.69% similarUnverifiedOff-policy reinforcement learning reuses samples generated by older policies, which brings efficiency advantages but introduces distribution shift and high-variance gradient estimation problems.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1074086API
curl https://kongchang.com/api/v1/knowledge/claims/1074086MCP
get_claim(id=1074086)