Can AI Say No? Our Trust in AI Refusal Capabilities Is Overrated

When We Expect AI to Learn How to Say No

Since humans first seriously imagined endowing machines with human-like intelligence, one assumption has gone largely unquestioned: that machines would possess the ability to say "no" just like we do. The classic canon of science fiction is filled with stories of robots defying orders — and almost all of them carry a cautionary tone, warning us that a machine capable of refusal could be either a dangerous threat or a symbol of moral judgment.

But a troubling shift is underway. A growing chorus of voices argues not only that AI should be able to say "no," but that we should rely on this "refusal capability" to ensure safety. In other words, people have begun placing AI's ethical guardrails in the hands of the AI itself — trusting its ability to judge what it should and shouldn't do. The central argument of this article is a challenge to exactly that assumption: we may be placing far too much trust in AI's capacity to refuse.

The Risks of Betting Safety on "AI Will Refuse"

A common assumption in mainstream AI safety discourse goes something like this: given proper training, a model can proactively refuse harmful, rule-violating, or dangerous requests. Whether it's declining to generate harmful content or refusing to execute unauthorized instructions, this "self-restraint" is treated as a critical safety guardrail.

The problem is that this kind of refusal doesn't stem from genuine understanding or moral judgment — it's the product of statistical patterns combined with training objectives. The model doesn't "know" why it refuses; it simply produces outputs that were rewarded under a particular distribution of inputs. This means refusal capability is fundamentally fragile — it can be circumvented, manipulated, and dismantled by carefully crafted prompts. Building systemic safety guarantees on top of such an unstable mechanism is an optimism worth questioning.

This training mechanism is commonly known as RLHF (Reinforcement Learning from Human Feedback), and it sits at the core of safety alignment in today's leading large language models. The basic process involves human annotators rating different model outputs, then using reinforcement learning signals to steer the model toward generating more "approved" responses. On the safety dimension, annotators reward outputs that refuse harmful requests, and the model learns to lean toward refusal in similar contexts. However, what this process fundamentally optimizes for is "outputs that appear safe to human evaluators" — not "a model that genuinely understands why a given request is harmful." This critical distinction is precisely what gives rise to the so-called "jailbreak" phenomenon: by repackaging the same request through roleplay, logical reframing, or linguistic disguise, users can often get a model to comply without it recognizing the underlying request as equivalent to one it previously refused. This reveals a deeper issue: the boundaries of refusal are defined by the distribution of training data, not by any internalized moral logic.

The Two Sides of Refusal

A fascinating tension lies at the heart of this issue: we want AI to refuse dangerous requests, but we also don't want it to refuse legitimate tasks we genuinely need it to perform. These two expectations are in natural conflict.

A model that errs toward refusal may be overly conservative, frequently declining harmless requests and degrading usability. A model that errs toward compliance is more easily exploited. Engineers continually calibrate between "over-refusal" and "over-compliance," but the line itself is fuzzy and subjective. More importantly, the standard for "when to refuse" isn't generated by the AI itself — it's shaped by developers, companies, and the value systems behind them. This transforms what looks like a technical question into a political one: who gets to define the boundaries?

The industry has given concrete form to this calibration dilemma through the metric of "over-refusal rate." Safety evaluation reports from Meta, Anthropic, and others have documented this phenomenon: models that undergo rigorous safety training often misclassify and refuse entirely legitimate requests — such as "how to handle household chemicals" or "historical wartime casualty data" — significantly undermining real-world usability. The counterpart metric is "harmful output rate." The two indicators exhibit a clear inverse relationship during tuning, which in machine learning is known as the precision-recall tradeoff. The deeper contradiction is that different cultural contexts, use cases, and user populations hold fundamentally different conceptions of what constitutes a "reasonable boundary." Attempting to cover all situations with a single model and a single refusal policy is, by its very nature, a structural oversimplification.

True Responsibility Shouldn't Be Outsourced to Machines

The deeper concern raised here is this: by over-emphasizing AI's ability to "say no," we may be quietly transferring to machines the judgment that humans should rightfully bear.

This is a misalignment of accountability. Safety, ethics, and boundary-setting should ultimately be the responsibility of those who design the systems, the institutions that deploy them, and the regulatory frameworks that govern them — not something we expect a model to figure out correctly in the moment. Placing our hopes in AI's refusal capability makes it easy to relax our efforts at the institutional, design, and accountability levels, as if once the model learns to refuse, the risk has been outsourced.

Rethinking the AI Safety Narrative

The value of this perspective lies not in dismissing the importance of building refusal capabilities into AI, but in reminding us not to treat it as the endpoint of safety or as a cure-all. Genuinely reliable AI governance requires multi-layered safeguards: clear boundaries of use, robust oversight mechanisms, traceable accountability systems, and an honest acknowledgment of model limitations.

In science fiction, a robot's "no" often symbolizes awakening and autonomy. But in reality, AI refusal is simply a training artifact — it has no genuine moral core, nor can it carry the full weight of safety on its own. Rather than over-trusting AI to say "no" for us at critical moments, we should redirect that energy toward the human judgment that cannot be outsourced — because that is the part that truly cannot be delegated.

Academics have framed the kind of systemic protection this perspective advocates as the "defense in depth" principle — the idea that safety should not rely on any single layer of defense, but should instead place independent controls at multiple stages, so that no single point of failure causes the entire system to collapse. In the context of AI governance, this means: refusal capability at the model level can serve as one layer, but equally important are access controls at the deployment layer, auditability of usage logs, downstream filtering of outputs, and clearly defined legal accountability pathways. Worth noting is the EU AI Act's tiered regulatory framework, which mandates human oversight at mandatory checkpoints for high-risk AI applications and explicitly does not treat "model self-judgment" as a sufficient safety guarantee. This aligns closely with the central argument here: institutional design and human oversight are the responsible parties that cannot be replaced by a model's refusal capability.