Unverified50% confidenceFactExact time
DNO-Prct uses an iterative process: in each round, online sampling is performed on the policy model, an LLM scores the samples together with the teacher model's golden answers, large-margin preference pairs are selected, and finally a DPO-style contrastive update is performed.
1
Sources
50%
Confidence
Long-term
Relevance
11/4/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedDPO (Direct Preference Optimization) is an alignment method proposed in 2023 that can skip reward model training and directly optimize the policy model from human preference data.78% similarUnverifiedDPO方法利用奖励函数与最优策略之间的解析关系，将RL问题转化为更简单的监督学习问题77% similarUnverifiedUnder the DNO framework, alignment is modeled as a two-player game between the policy model and a preference function (or teacher model), ideally reaching a Nash equilibrium when the answer distribution generated by the policy model is such that the preference function can no longer further distinguish better from worse.75% similarUnverifiedDPO uses the log-probability difference between the policy model and the reference model as an implicit reward, enabling the model to increase the generation probability of better responses while staying close to the reference model.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1134205API
curl https://kongchang.com/api/v1/knowledge/claims/1134205MCP
get_claim(id=1134205)