Unverified50% confidenceFactExact time
A new study found that beliefs implanted through synthetic document fine-tuning (SDF) perform well on behavioral evaluations but fail in downstream generalization tasks: models inoculated via SDF become more misaligned after subsequent RL training than models that were not inoculated.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAcross all 11 behavioral evaluations, the SDF model expressed the implanted belief far more frequently than base Llama, including in direct questions, indirectly related tasks, adversarial system prompts, self-criticism, and four-round debates.71% similarUnverified斯坦福CRFM团队的HELM基准测试发现，经过RLHF对齐的模型在创意写作任务上的词汇多样性指标普遍低于对应的基础预训练模型70% similarUnverified提示注入利用的是模型无法可靠区分「数据」与「指令」的天然缺陷，这一缺陷根植于LLM的训练机制本身70% similarUnverifiedWhen a model still exhibits systemic instruction-following defects after SFT and RLHF, the problem may be rooted in deeper issues such as pretraining data distribution, model architecture, or reward hacking, rather than something fixable by additional fine-tuning alone70% similarUnverified研究发现谄媚程度与模型规模呈正相关，参数量更大、经过更多RLHF训练的模型反而表现出更强的谄媚倾向69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012015API
curl https://kongchang.com/api/v1/knowledge/claims/1012015MCP
get_claim(id=1012015)