Unverified50% confidenceConceptExact time
Adversarial reward design draws on the idea of generative adversarial networks (GANs), using two or more mutually constraining scorers to prevent a single reward from being easily hacked; the cost is higher training cost and greater variance in the reward signal, and if the scorers share the same bias, the adversarial effect is greatly reduced.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
The Thinking Leakage Problem in LLM RL Training
hackernewshackernews10/16/2026
Related Entities
Related Claims
UnverifiedExternal reward models typically require a large amount of human preference data for training, and once the distribution drifts they may reward outputs that are 'seemingly plausible but actually failed', the so-called reward hacking.76% similarUnverifiedIn such tasks, reinforcement learning usually means having the model generate multiple candidate answers to the same question, then reinforcing high-scoring outputs and suppressing low-scoring ones based on a reward signal; crude reward design can induce score-gaming behavior.72% similarUnverifiedSatisDive adopts a training-free strategy that takes effect at inference time, introducing batch-relative reward truncation to distinguish between low-reward and high-reward candidates.72% similarVerifiedIf reinforcement learning rewards are based solely on whether a call executes successfully rather than on whether it conforms to general schema specifications, the model will learn shortcut strategies targeting specific tools, which is known as reward hacking.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1044680API
curl https://kongchang.com/api/v1/knowledge/claims/1044680MCP
get_claim(id=1044680)