Unverified50% confidenceTradeoffExact time
Current mainstream alignment techniques such as RLHF are often deeply coupled with improvements in model capability, making it technically difficult to fix alignment strictly without also improving capability.
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedPachocki believes that current alignment techniques such as RLHF may have fundamental limitations when faced with superintelligent systems.80% similarUnverified2023年多项研究证明，即使经过RLHF对齐的模型也难以完全抵抗精心设计的间接提示词注入77% similarUnverified仅靠微调阶段的RLHF或许永远无法产生真正鲁棒的安全对齐76% similarVerifiedTraditional RLHF methods have a fundamental limitation of 'reward hacking,' where models may learn to perform well in evaluation scenarios but fail in out-of-distribution scenarios72% similarUnverified研究表明RLHF在模型残差流中留下的安全拒绝行为呈现低秩线性结构，主要分布在少数几个主方向上，因此易被正交投影移除71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1000065API
curl https://kongchang.com/api/v1/knowledge/claims/1000065MCP
get_claim(id=1000065)