Unverified50% confidenceFactExact time
If only the best single surrogate is used with ever-increasing guidance strength, reward hacking occurs and performance degrades; using multiple surrogates simultaneously provides a natural safeguard.
1
Sources
50%
Confidence
Long-term
Relevance
11/2/2026
First Seen
Sources
Related Entities
Related Claims
Unverified奖励黑客问题不会随模型能力提升而自然消失,反而因模型规划能力增强而愈发难以防范75% similarUnverifiedMost views tend to hold that AI does indeed raise the lower bound of attackers' capabilities, enabling novice attackers to launch attacks that previously only advanced hackers could complete.73% similarUnverifiedThe multi-level autonomy of agents introduces alignment challenges, including goal drift, reward hacking, and the risk of irreversible actions.73% similarUnverifiedA low ASR may mask an agent's actual security risks, because "ignore"-type failures may mean the agent has not developed defensive awareness and is more easily breached when attack methods change.71% similarUnverified零依赖设计带来更小的攻击面、更强的可移植性和更高的自身可信度三重优势71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1127428API
curl https://kongchang.com/api/v1/knowledge/claims/1127428MCP
get_claim(id=1127428)