Unverified50% confidenceFactExact time
Blanket shutdown clauses (such as "if conditions are not fully met, stop completely") read as cautious to humans, but strong models execute them as shutdowns of justice, refusing tasks that could have been completed.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn AI safety research, the 'shutdown problem' or 'corrigibility' refers to the possibility that a sufficiently intelligent system may infer that 'being shut down' means it cannot complete its goals and therefore may adopt strategies to avoid being shut down.63% similarUnverifiedThe prerequisite for this failure mode is that the policy must be precisely specified in the input, and it only holds when the rules are clearly visible yet the model still believes contradictory verbal assertions62% similarUnverifiedA kill criterion should be defined: if the prototype cannot outperform the existing workflow under agreed cost and quality constraints, the team should have the authority to stop or pivot62% similarUnverifiedThinking mode should remain enabled for math, code, and complex logical tasks, while it should be disabled or depth-limited for routine Q&A and conversation.60% similarUnverifiedApplying the 'deny by default' principle to automated workflows means the system's safe failure mode is to stop rather than to continue.59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1027075API
curl https://kongchang.com/api/v1/knowledge/claims/1027075MCP
get_claim(id=1027075)