Unverified50% confidenceOpinionExact time
Excessively thorough SFT may harm the exploration space for subsequent RL, and a perfectly converged SFT model is not necessarily the best starting point for RL.
1
Sources
50%
Confidence
Long-term
Relevance
9/7/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSFT fine-tuning is effective but causes a decline in general knowledge in small models—i.e., the phenomenon of catastrophic forgetting—due to the prominent capacity bottleneck of small models.72% similarUnverifiedIn the standard SFT process, the model continuously expends gradients on sequences that are already well-fitted, causing the output distribution to narrow and entropy to decrease, thereby compressing the exploration space for subsequent RL.71% similarUnverifiedThe core challenge of offline RL is distributional shift, which can lead to extrapolation errors that cause the learned policy to perform far worse than expected at deployment.65% similarUnverifiedRRF对各路检索的原始分数分布不敏感，相比简单的线性加权在混合检索场景中更加鲁棒64% similarUnverified仅靠微调阶段的RLHF或许永远无法产生真正鲁棒的安全对齐62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/873213API
curl https://kongchang.com/api/v1/knowledge/claims/873213MCP
get_claim(id=873213)