Unverified50% confidenceFactExact time
Traditional SFT only maximizes the likelihood of the labeled data and lacks a mechanism for preserving pretrained knowledge, so its performance is limited in small-data scenarios.
1
Sources
50%
Confidence
Long-term
Relevance
11/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe number of trainable parameters in PEFT is usually less than 1% of the total parameters, greatly reducing the risk of overfitting and forgetting in small-data scenarios.77% similarUnverifiedThe paper proves that starting from the same perfectly fitted checkpoint, SFT's forgetting over a finite training interval has a lower bound that is strictly greater than zero.73% similarUnverifiedContext confusion produces narrow misalignment, where misalignment is concentrated in specific domains semantically close to the training data, rather than a broad degradation of the model's overall behavior.73% similarUnverifiedSFT fine-tuning is effective but causes a decline in general knowledge in small models—i.e., the phenomenon of catastrophic forgetting—due to the prominent capacity bottleneck of small models.73% similarUnverifiedLong-tail scenarios refer to edge cases that occur with extremely low statistical frequency but must be handled correctly by the system, and are naturally scarce in training data73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1182998API
curl https://kongchang.com/api/v1/knowledge/claims/1182998MCP
get_claim(id=1182998)