SFT
Supervised Fine-Tuning (SFT) is a machine learning technique that involves training a pre-trained model on labeled instruction-response pair data in a supervised manner, enabling the model to follow instructions and generate expected outputs. In the alignment pipeline for large language models, SFT is a key step for equipping models with conversational and instruction-following capabilities. During training, loss is typically computed only on the model's response portion to guide the model toward correct response patterns.
Core Facts
Modern large language models typically go through three training stages: pre-training, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF).
The RLHF pipeline consists of three steps: training a reward model to predict human preference scores, adjusting language model parameters using the PPO reinforcement learning algorithm, and using KL divergence constraints to prevent the model from deviating from its original language capabilities.
当代前沿模型的训练方法包括RLHF（基于人类反馈的强化学习）及其后继者RLAIF和过程奖励模型（PRM）
Timeline (last 90 days)
Low-quality trajectories in supervised fine-tuning reinforce erroneous behavioral patterns, while high-quality trajectories from real verifiable environments can teach the model more reliable development strategies.
On the public sequential recommendation benchmark KuaiRand, EWM surpassed SFT and LaRe in the offline setting and outperformed the A2C algorithm in the online setting.
Post-training is a critical phase between the completion of large language model pretraining and actual usability, encompassing technical approaches such as supervised fine-tuning (SFT), instruction alignment, and RLHF/DPO.
This method uses SFT rather than complex alignment techniques such as RLHF or DPO
The classifier was trained using only supervised fine-tuning (SFT), public datasets, and two simple tricks, without relying on closed-source data or expensive custom pipelines
A middle-ground approach that teams evaluating classification requirements can adopt is: use public datasets with SFT for low-cost fine-tuning, validate the results, and then decide whether to invest more resources
For classification tasks with discrete and closed label spaces, the training signal from SFT is already clear enough, and introducing reinforcement learning would instead increase the difficulty of hyperparameter tuning and training instability
There are typically three technical approaches to achieving specialized intelligence: supervised fine-tuning (SFT), retrieval-augmented generation (RAG), and pre-training from scratch.
Technical difficulty exists on a gradient: fine-tuning < SFT < pre-training, and the recommended learning order is to start with fine-tuning first.
3,189 successful rollouts were collected from Qwen3.8-27B for supervised fine-tuning (imitation learning), and performance stagnated at 47.5%, below the results of two rounds of RL training.
40 more timeline events
All Facts (20)
Modern large language models typically go through three training stages: pre-training, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF).
90%VerifiedThe RLHF pipeline consists of three steps: training a reward model to predict human preference scores, adjusting language model parameters using the PPO reinforcement learning algorithm, and using KL divergence constraints to prevent the model from deviating from its original language capabilities.
90%Verified当代前沿模型的训练方法包括RLHF（基于人类反馈的强化学习）及其后继者RLAIF和过程奖励模型（PRM）
80%Verified"Post-training" is the alignment and capability enhancement phase following pre-training, and typically includes SFT, RLHF, and reinforcement learning based on process reward models (such as GRPO and DAPO).
80%VerifiedSupervised Fine-Tuning (SFT) trains the model using 'prompt-response' example pairs, teaching the model to respond in a conversational format, thereby making outputs more coherent.
75%VerifiedLarge model training goes through three core stages: pre-training, supervised fine-tuning (SFT), and preference alignment.
75%VerifiedDomain fine-tuning typically first injects domain knowledge through supervised fine-tuning (SFT), then aligns with user preferences through reinforcement learning from human feedback (RLHF).
65%VerifiedDuring SFT training, the model only computes loss on the assistant portion (loss masking technique), meaning it only learns how to answer, not how to ask questions.
65%Unverified大模型核心原理包括Transformer架构、预训练、SFT监督微调、RLHF人类反馈强化学习等关键概念
95%Unverified继续预训练（CPT）后通常还需要再做一轮SFT才能恢复模型的对话能力，因为CPT可能会削弱模型在SFT阶段学到的指令遵循能力
85%UnverifiedSpecialized models trained specifically on cybersecurity data have the potential to surpass general-purpose models in vulnerability identification accuracy and false positive rates.
70%UnverifiedModels trained with reinforcement learning (RL) outperform models trained solely with SFT on Agent tasks.
70%Partially Verified从预训练检查点到可商用产品需经历监督式指令微调（SFT）和基于人类反馈的强化学习（RLHF）两个后训练阶段
65%UnverifiedSFT fine-tuning is effective but causes a decline in general knowledge in small models—i.e., the phenomenon of catastrophic forgetting—due to the prominent capacity bottleneck of small models.
60%UnverifiedA typical post-training pipeline is: pre-training → supervised fine-tuning (SFT) → preference optimization (RLHF / DPO / GRPO).
60%UnverifiedThe emergence of BERT and GPT in 2018 initiated the pre-training + fine-tuning paradigm
60%UnverifiedThe learning rate during the SFT stage is typically 1–2 orders of magnitude lower than during pre-training, to avoid disrupting the general knowledge learned during the pre-training stage.
60%UnverifiedLow-quality trajectories in supervised fine-tuning reinforce erroneous behavioral patterns, while high-quality trajectories from real verifiable environments can teach the model more reliable development strategies.
50%UnverifiedOn the public sequential recommendation benchmark KuaiRand, EWM surpassed SFT and LaRe in the offline setting and outperformed the A2C algorithm in the online setting.
50%UnverifiedPost-training is a critical phase between the completion of large language model pretraining and actual usability, encompassing technical approaches such as supervised fine-tuning (SFT), instruction alignment, and RLHF/DPO.
50%