Ambient SFT
A data-efficient supervised fine-tuning (SFT) framework. The fine-tuned model matches the predictions of a frozen pretrained model on a high-temperature-softened, coarse-grained distribution, preserving pretrained knowledge through regularization and mitigating overfitting and catastrophic forgetting in small-data SFT.
Timeline (last 90 days)
The Ambient SFT framework mitigates the overfitting and catastrophic forgetting problems of small-data SFT by having the fine-tuned model match the pretrained model's high-temperature conditional distribution at a coarse granularity.
In Ambient SFT, the temperature hyperparameter T controls the degree of trust in the pretrained model: when T tends to infinity, the method degenerates to standard SFT; when T=1, it becomes knowledge distillation.
The paper evaluates Ambient SFT on domain adaptation, instruction fine-tuning and reasoning tasks, and the results show that the method outperforms standard SFT in all these scenarios, with an average relative improvement of 21%.
Ambient SFT's improvement does not come from additional labeled data but is achieved by fully exploiting the prediction distribution of the frozen pretrained model.
In knowledge distillation and Ambient SFT, the loss function usually consists of a task loss and a KL divergence regularization term: the former pulls the model output closer to the hard labels, while the latter constrains the model distribution not to deviate too far from the teacher distribution.
The temperature T plays a key role in Ambient SFT: at high temperatures the softmax output tends toward uniformity, and the KL term's constraint on the distribution becomes coarse-grained, requiring only that the model stay consistent with the teacher on larger class groupings.
For practitioners, Ambient SFT offers a simple and implementable improvement: introducing a temperature-matching term into the standard SFT loss yields more robust models in small-data scenarios.
Future directions for Ambient SFT may extend to more modalities and tasks, or combine with parameter-efficient fine-tuning techniques to further lower the barrier to fine-tuning.
All Facts (8)
The Ambient SFT framework mitigates the overfitting and catastrophic forgetting problems of small-data SFT by having the fine-tuned model match the pretrained model's high-temperature conditional distribution at a coarse granularity.
50%UnverifiedIn Ambient SFT, the temperature hyperparameter T controls the degree of trust in the pretrained model: when T tends to infinity, the method degenerates to standard SFT; when T=1, it becomes knowledge distillation.
50%UnverifiedThe paper evaluates Ambient SFT on domain adaptation, instruction fine-tuning and reasoning tasks, and the results show that the method outperforms standard SFT in all these scenarios, with an average relative improvement of 21%.
50%UnverifiedAmbient SFT's improvement does not come from additional labeled data but is achieved by fully exploiting the prediction distribution of the frozen pretrained model.
50%UnverifiedIn knowledge distillation and Ambient SFT, the loss function usually consists of a task loss and a KL divergence regularization term: the former pulls the model output closer to the hard labels, while the latter constrains the model distribution not to deviate too far from the teacher distribution.
50%UnverifiedThe temperature T plays a key role in Ambient SFT: at high temperatures the softmax output tends toward uniformity, and the KL term's constraint on the distribution becomes coarse-grained, requiring only that the model stay consistent with the teacher on larger class groupings.
50%UnverifiedFor practitioners, Ambient SFT offers a simple and implementable improvement: introducing a temperature-matching term into the standard SFT loss yields more robust models in small-data scenarios.
50%UnverifiedFuture directions for Ambient SFT may extend to more modalities and tasks, or combine with parameter-efficient fine-tuning techniques to further lower the barrier to fine-tuning.
50%