ORPO
An alignment method variant in the DPO family that merges supervised fine-tuning with preference alignment into a single training process, requiring no separate reference model.
Timeline (last 90 days)
LLaMA-Factory integrates training algorithms such as incremental pre-training, instruction-supervised fine-tuning (SFT), reward model training (Reward Model), PPO, DPO, KTO, and ORPO.
ORPO merges SFT and preference alignment into a single stage.
DPO variants such as KTO, ORPO and SimPO further relax the requirements for paired data or reference models.
All Facts (3)
LLaMA-Factory integrates training algorithms such as incremental pre-training, instruction-supervised fine-tuning (SFT), reward model training (Reward Model), PPO, DPO, KTO, and ORPO.
50%UnverifiedORPO merges SFT and preference alignment into a single stage.
50%UnverifiedDPO variants such as KTO, ORPO and SimPO further relax the requirements for paired data or reference models.
50%