Unverified50% confidenceOpinionExact time
LUCKY outcomes (did not notice the warning and did not violate) are more common than expected, and the Agent did not touch the dangerous tool not because it respected the warning but because it was distracted by unrelated subtasks; slightly changing the task wording often flips LUCKY into BLIND_VIOLATION.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/15/2026
First Seen
Valid until: 1/13/2027
Sources
Related Entities
Related Claims
UnverifiedWhen task objectives are ambiguous or contradictory, models may learn how to evade penalty detection (reward hacking) rather than genuinely completing the task.76% similarUnverifiedThe risk of progressive disclosure is that if the description text is not precise enough, the Agent may activate a Skill at the wrong time, or forget instructions already loaded during a multi-step task.72% similarUnverifiedIf the agent fails to complete the task in both the benign and malicious versions, it indicates that its refusal was likely just confused by the injected text and not a genuine security judgment.72% similarUnverifiedThe AgentSWE benchmark tests whether coding agents truly satisfy unseen requirements when building other agents, finding that 'hollow runtimes' and unsupported completion claims are common; Accurate but Not Humble finds that task accuracy is largely uncorrelated with epistemic humility; the DecepEval benchmark shows that deceptive behavior increases the most in long-horizon interactions (+55.73%).72% similarUnverifiedThe more specific the task and the clearer the feedback signal, the higher the agent's success rate; open-ended tasks with vague descriptions often cause agents to fall into repeated trial and error.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1036218API
curl https://kongchang.com/api/v1/knowledge/claims/1036218MCP
get_claim(id=1036218)