Unverified50% confidenceTradeoffExact time
DPO隐含假设偏好数据服从Bradley-Terry模型，当偏好具有非传递性时可能产生不一致行为
1
Sources
50%
Confidence
Long-term
Relevance
8/6/2026
First Seen
Sources
Related Claims
UnverifiedPreference alignment with the DPO algorithm requires preparing both positive examples (preferred responses) and negative examples (dispreferred responses).72% similarUnverifiedIn RLHF, if preference data is of low quality or contains systematic bias, the reward model will learn incorrect signals, causing the model's behavior to deviate from expectations.68% similarUnverified自我偏好偏见指裁判模型倾向偏好由同系列模型生成的输出67% similarUnverifiedDPO的核心洞见是语言模型本身可视为隐式奖励函数，因此可直接对偏好对进行对比学习，绕过显式奖励模型66% similarUnverified循环论证偏差指当特征本身隐含标签信息时模型看似准确率极高实则用答案预测答案66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/695772API
curl https://kongchang.com/api/v1/knowledge/claims/695772MCP
get_claim(id=695772)