DPO (Direct Preference Optimization) is an alignment method proposed in 2023 that can skip reward model training and directly optimize the policy model from human preference data.