RLHF is the core training paradigm for aligning mainstream large language models (such as GPT-4 and Claude) with human preferences, by collecting human preference annotations to train a reward model and then fine-tuning the generative model.