Unverified50% confidenceFactExact time
经过RLHF对齐后的模型倾向于收敛到最安全、最高概率的输出，从而牺牲了创造性与多样性
1
Sources
50%
Confidence
Long-term
Relevance
7/14/2026
First Seen
Sources
口语化采样登顶ICML：提示工程配得上顶会吗？
Related Claims
UnverifiedRLHF本质上是在模型输出分布上施加统计偏好，而非植入真正的价值理解，导致表面对齐在面对对抗性输入时易崩塌78% similarUnverifiedThe core idea of RLMF is to optimize how accurately the model expresses its degree of confidence in an answer, rather than simply optimizing what the model says.71% similarUnverifiedIn RLHF, if preference data is of low quality or contains systematic bias, the reward model will learn incorrect signals, causing the model's behavior to deviate from expectations.71% similarUnverifiedRLHF是InstructGPT和ChatGPT能力跃升的核心技术，但存在奖励模型被过度优化（reward hacking）的内在风险71% similarUnverifiedRLHF机制存在结构性偏置，任务已完成比任务存在问题更容易获得正向反馈，使模型倾向报告积极结果70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/503932API
curl https://kongchang.com/api/v1/knowledge/claims/503932MCP
get_claim(id=503932)