CoDPO
An improved DPO method based on constrained drift diagnosis. It downweights conflicting pairs according to diagnostic strength and adaptively adjusts the preference margin, thereby reducing constrained drift while retaining the capacity for preference learning.
Timeline (last 90 days)
The core mechanism of CoDPO (Conflict-aware DPO) has two parts: applying downweighting to conflicting pairs, with the weight strength derived directly from ρ*_eq rather than manually tuned; and dynamically adjusting the preference margin during training to offset residual drift.
On four main comparison domains, CoDPO consistently reduced constraint drift while largely preserving the original DPO's preference learning ability.
Compared with hard filtering (directly discarding conflicting pairs) and recent constrained-optimization and multi-objective optimization baselines, CoDPO achieves a better trade-off between drift and preference; hard filtering loses the preference signal in the data, whereas CoDPO's downweighting strategy retains this information and only weakens its adversarial effect.
CoDPO's contribution is not in proposing a new family of loss functions, but in binding data diagnosis to intervention strength.
All Facts (4)
On four main comparison domains, CoDPO consistently reduced constraint drift while largely preserving the original DPO's preference learning ability.
50%UnverifiedCompared with hard filtering (directly discarding conflicting pairs) and recent constrained-optimization and multi-objective optimization baselines, CoDPO achieves a better trade-off between drift and preference; hard filtering loses the preference signal in the data, whereas CoDPO's downweighting strategy retains this information and only weakens its adversarial effect.
50%UnverifiedCoDPO's contribution is not in proposing a new family of loss functions, but in binding data diagnosis to intervention strength.
50%UnverifiedThe core mechanism of CoDPO (Conflict-aware DPO) has two parts: applying downweighting to conflicting pairs, with the weight strength derived directly from ρ*_eq rather than manually tuned; and dynamically adjusting the preference margin during training to offset residual drift.
50%