DPO uses a mathematical transformation to equivalently rewrite the "reward maximization + KL constraint" objective as a simple classification loss, requiring no reward model and no reinforcement learning loop, and alignment can be completed with supervised learning alone.