Unverified50% confidenceFactExact time
The key mechanism of RM-OPD is applying token-level dense supervision on critiques generated by the student itself, making the supervision distribution consistent with the student's current policy distribution and alleviating the training-inference distribution mismatch problem.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedOnline policy distillation (OPD) has the student model generate its own training responses, which are then supervised by the teacher model.69% similarUnverifiedTraining-time alignment (such as RLHF and DPO), which is often contrasted with inference-time alignment, writes value preferences directly into the weights, offering greater stability but poor flexibility, and making it difficult to dynamically adjust for individual users.65% similarUnverifiedBehavioral cloning suffers from distribution shift; improved algorithms such as DAgger address this by having the learning agent continuously interact with the expert to expand the training data.65% similarUnverifiedCompared to conventional teacher forcing, GTF can reduce exposure bias (the distribution mismatch caused by relying on ground-truth data during training and on the model's own predictions during inference).63% similarUnverifiedSecurity alignment typically includes techniques such as RLHF (Reinforcement Learning from Human Feedback) and DPO (Direct Preference Optimization).63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1044463API
curl https://kongchang.com/api/v1/knowledge/claims/1044463MCP
get_claim(id=1044463)