AI Agent Tool Permission Design: Why Limits Belong Outside the Prompt

When configuring tools for an AI Agent, most people's first instinct is to write clearly in the prompt: "don't do X" and "only do Y." But Paris-based full-stack developer Amine Affif proposes a completely opposite approach: set the limits at the moment the process starts, not in the prompt.

Prompts Are Requests, Not Constraints

Affif's core judgment is straightforward: a prompt is essentially a request. The model can choose to ignore it, or it can be steered off course by malicious text—especially when the Agent needs to read external content it doesn't control.

This aligns with how traditional software engineering treats third-party dependencies: don't trust, but isolate. You wouldn't feel safe just because a dependency library's documentation says "do not access the network"—you'd cut off its network permissions at the runtime level. Affif believes Agent tool permissions should be handled with the same logic.

The Agent he built himself uses a closed tool allowlist: twelve read tools pre-permitted, fifteen write tools denied. The result is that this Agent cannot write files or send messages—even if it "wants" to.

Three Supplementary Layers of Runtime Isolation

Beyond the hard switches of the tool allowlist, Affif layers on three additional safeguards:

Anti-injection prompt section. Because the Agent reads uncontrolled text, a dedicated section of the prompt is set aside to counter injection attacks. This isn't a panacea—it's an added threshold of interference at the model level.

Mandatory adversarial checking. Before every output, the Agent must go through a round of adversarial review, similar to having the model "attack itself" to surface potential problems.

Human verification. Any content requires human confirmation before publication. This layer may seem inefficient, but it keeps the final decision in human hands.

One detail worth noting: these three protective layers do not substitute for runtime isolation. Prompt-level defenses can be bypassed; runtime isolation is the wall that holds as a last resort.

Prompt Injection refers to attackers hiding malicious instructions in external content the Agent reads (such as web pages, documents, or emails), inducing the model to deviate from its intended task. It's dangerous because the model struggles to distinguish "data" from "instructions"—a piece of text being read may appear to the model to have the same authority as the system prompt. This resembles traditional SQL injection: both treat untrusted input as executable commands. Defense approaches typically fall into two layers—model-level detection and filtering (such as anti-injection prompt sections), and system-level permission narrowing (such as runtime isolation), the latter of which does not depend on model judgment and is therefore more reliable.

Adversarial Checking draws on the ideas of adversarial example research: having another model instance or the same model examine output from an "attacker's perspective," looking for logical flaws, factual errors, or security risks. Common implementations include self-critique, red-teaming prompts, and multi-round debate. It is a probabilistic safeguard that increases the rate at which problems surface but cannot guarantee exhaustive coverage of all risks, making it suitable as a supplementary layer alongside runtime isolation rather than a replacement.

An Agent That Can Act Obeys Its Environment, Not Politeness

Affif's reasoning is worth chewing on repeatedly: an Agent that can act does what the environment allows it to do, not what it is politely asked to do. If an action is physically impossible, there is no room for "trust" to matter.

This statement pinpoints a common misconception in current Agent safety discussions: treating reliability problems as alignment problems. Writing longer system prompts and adding more "do not" clauses is essentially using probabilistic means to counter deterministic risks. Runtime permission control, by contrast, turns impossible actions into unexecutable actions—no model cooperation needed, no "understanding" required.

Runtime Isolation differs fundamentally from prompt constraints in where enforcement happens: the former cuts off capabilities at the process, container, or system-call level, while the latter merely expresses intent at the token level. Common engineering techniques include: configuring the process's accessible file system and network according to the principle of least privilege; restricting system calls with mechanisms like seccomp and AppArmor; isolating execution environments with containers or sandboxes; and rejecting unauthorized calls directly at the framework level via tool allowlists/denylists. Its core advantage is determinism—no matter what the model outputs, unauthorized actions are physically impossible. The cost is reduced flexibility, requiring you to think through exactly what capabilities the Agent truly needs at the design stage.

Speed Isn't the Point—Control Is

In his article, Affif admits that what concerns him is not the speed boost tools bring, but how to constrain those tools. This ability to constrain is the precondition that makes it safe to move fast—otherwise you're dragging unread code into the repository.

This perspective has practical value for engineering teams: when an Agent is given write permissions, where does code review end? When an Agent can send messages, who is responsible for their content? Runtime isolation's answer is—don't let these problems happen, rather than assigning blame after they do.

For developers bringing Agents into production environments, Affif's approach offers a lightweight but clear paradigm: first define the boundaries of what tools the Agent can touch, then talk about prompt optimization. Reverse the order, and the risk spirals out of control.

One-line Summary

AI Agent tool permissions should be hard-isolated at runtime rather than constrained through prompts.

Paragraph Summary

The article presents Paris-based developer Amine Affif's approach to Agent security: prompts are merely requests, not constraints, and models may ignore them or be steered by malicious text. He therefore uses a closed tool allowlist that pre-permits read tools and denies write tools at process startup, making dangerous actions physically impossible for the Agent. On top of this, he layers three supplements: anti-injection prompts, mandatory adversarial checking, and human verification. The core argument: an Agent that can act obeys environmental limits, not polite requests—mistaking reliability problems for alignment problems and countering deterministic risks with longer prompts is the wrong direction. For engineering teams, defining tool boundaries before optimizing prompts is the precondition for advancing Agent adoption safely and quickly.