Unverified50% confidenceSolutionExact time
If the amount of fine-tuning data is limited, directly unfreezing all parameters for fine-tuning will almost inevitably trigger forgetting; a more reliable approach is to first freeze the backbone network and train only the final output head or adaptation layers.
1
Sources
50%
Confidence
Long-term
Relevance
11/9/2026
First Seen
Sources
Related Entities
Related Claims
Unverified冻结基准可能因数据泄露被纳入模型训练数据而随时间失去区分力,这是一个权衡77% similarUnverifiedGradual Unfreezing progressively unfreezes and trains lower-layer parameters starting from the top layer, which can prevent Catastrophic Forgetting.74% similarUnverifiedGreedy decoding at temperature zero further suppresses the probability of a model outputting ESCALATE, because models tend to select the answer pattern that appeared most frequently during training, and escalation is inherently rare in instruction fine-tuning data.73% similarUnverifiedOver-reliance on the fine-tuned part during fine-tuning may be precisely the root cause of rare-class capabilities being obscured; directing the model's computation moderately toward the pretrained components may be a more robust fine-tuning route.73% similarVerifiedThe core idea of PEFT is to not update all parameters, but instead introduce a small number of trainable adapter parameters (such as LoRA's low-rank matrix decomposition), reducing VRAM and computation requirements while retaining performance close to full fine-tuning.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1149858API
curl https://kongchang.com/api/v1/knowledge/claims/1149858MCP
get_claim(id=1149858)