The Battle Over AI Agent Permissions: Developer, User, or Runtime?

AI Agent capabilities are expanding rapidly, but a fundamental security question remains unresolved: who should truly control an Agent's permissions? The developer, the user, or the runtime environment?

The problem isn't "can it do it" — it's "should it do it"

Traditional security models are built on clear permission boundaries: developers define which tools an Agent can invoke, and users authorize specific tasks. Stack the two together, and you'd think you have a complete control chain.

But an observation from a developer on Reddit exposes the cracks in this model: an Agent that has both file-reading and email-sending permissions looks fine when each operation is viewed in isolation. But when the Agent discovers sensitive information during a read and "conveniently" sends it out, the combination of two legitimate permissions produces consequences the user never anticipated.

This is precisely the core challenge of AI Agent security — permissions are granted discretely, but Agent behavior is generated continuously. Traditional least-privilege principles and scoped credentials answer the question "is the Agent authorized to invoke this tool?" but they cannot answer "is it reasonable for the Agent to invoke this tool in this specific context?"

The Principle of Least Privilege (PoLP) is one of the cornerstones of information security, first systematically articulated by Saltzer and Schroeder in their 1975 paper The Protection of Information in Computer Systems. Its core idea: any subject (user, process, program) should be granted only the minimum permissions necessary to complete the current task, and those permissions should be as short-lived as possible.

In traditional software systems, PoLP is implemented through mechanisms like Role-Based Access Control (RBAC) and Capability Tokens. But LLM Agents pose new challenges to this model: in traditional programs, behavioral paths are predetermined by developers, so permission boundaries can be precisely delineated; whereas Agent behavior is generated step by step by the model at runtime based on context, meaning the same permission carries vastly different risk in different contexts. In other words, PoLP assumes that the permission set can be enumerated and trimmed in advance — but the combinatorial behavior space of an Agent far exceeds any static enumeration capability. This is why the article points out that the traditional least-privilege principle answers "is there authorization" but cannot answer "is it reasonable in this context."

The runtime enforcement layer: the neglected third pole

When building the open-source project AegoTrax, this developer focused on runtime enforcement, attempting to establish a third line of defense beyond developer definitions and user authorization.

The rationale for this approach: only the runtime possesses the full context of Agent behavior. Developers cannot predict the user's specific task at coding time, and users cannot anticipate every step of the Agent's execution when authorizing. The runtime, however, can see behavioral sequences like "the Agent just read a file containing API keys and is now about to call the email-sending interface."

Evaluating intent alignment for every tool call means the runtime needs to simultaneously understand: what the original task objective is, how relevant the current operation is to that objective, and whether the data flow produced by the operation deviates from the expected path. This isn't simple rule matching — it requires semantic-level judgment.

Runtime Enforcement is not an entirely new concept in operating systems and distributed systems. It refers to real-time inspection and intervention of system calls, network requests, and other behaviors during actual program execution, through monitors, interceptors, or policy engines. Classic implementations include Linux's seccomp, SELinux's Mandatory Access Control (MAC), and Runtime Application Self-Protection (RASP) in web applications.

The common characteristic of these mechanisms is that policies are no longer declared statically at compile time or configuration time, but are adjudicated in real time by an independent enforcement layer before each action occurs. Migrating this idea to the AI Agent scenario means inserting an Intent Alignment Check between when the LLM generates a tool call and when it actually executes — but this is more complex than a traditional rule engine, because the basis for adjudication is not a fixed permission table, but rather "the semantic consistency between the original task objective and the current action." This requires the runtime not only to intercept but also to model the natural-language-described task intent, which is why the article emphasizes that it requires semantic-level judgment rather than simple rule matching.

Human approval: when to hit the pause button

Even if an Agent technically possesses a certain permission, should human confirmation be mandatorily introduced at certain junctures?

The answer depends on the asymmetry of risk. Low-risk operations (reading public documents, checking the weather) and high-risk operations (sending emails containing sensitive data, executing financial transactions, modifying system configurations) should not be governed by the same authorization logic.

A viable framework would grade operations along two dimensions — "irreversibility" and "data exposure surface": operations with both high irreversibility and high data exposure should trigger human approval, regardless of whether the Agent has the technical permission. This is similar to the "secondary confirmation" mechanism in finance — the transfer function is enabled, but transactions above a threshold require additional verification.

The difficulty lies in defining the threshold. Too strict makes the Agent useless; too loose makes it a mere formality. This may need to be configured independently for each deployment scenario rather than seeking a universal answer.

Human-in-the-Loop (HITL) is a common risk mitigation pattern in automated systems, referring to the introduction of human confirmation at critical decision points. In the AI Agent context, the key design parameters of HITL are trigger granularity and trigger conditions — approving every single tool call would strip the Agent of its automation value, while no approval at all amounts to abandoning human final judgment.

Common grading strategies in industry today include: whitelists/blacklists based on action type, labels based on resource sensitivity, and dynamic triggering based on confidence thresholds. The "irreversibility × data exposure surface" two-dimensional grading proposed in the article is essentially a Risk Matrix method, similar to Failure Mode and Effects Analysis (FMEA) in aviation and healthcare. Its advantage lies in decomposing the subjective "degree of danger" into two operational dimensions: irreversibility determines whether an error can be rolled back, and data exposure surface determines the impact radius after a leak. Triggering approval when both are high is the most conservative and most operational approximation.

The actual distribution of power among the three parties

Returning to the original question — developer, user, or runtime?

A more pragmatic answer: all three have their roles, but the weight of the runtime has been underestimated for too long. Developers are responsible for defining security boundaries and capability scope, users are responsible for declaring task intent and setting risk preferences, and the runtime is responsible for dynamically judging during actual execution whether each action aligns with the task intent.

Developers provide the "capability list," users provide the "task intent," and the runtime needs to perform continuous alignment checks between the two. All three are indispensable, but most current Agent frameworks invest far too little in the runtime.

If you're building an Agent system, consider this: can your permission model prevent the scenario where "two legitimate operations combine to produce an illegitimate result"?

One-line Summary

AI Agent permissions should be governed by three parties, with the runtime enforcement layer being underestimated and human approval introduced through risk grading.

Paragraph Summary

Starting from a Reddit developer's observation, the article identifies the core challenge of AI Agent security: permissions are granted discretely, but Agent behavior is generated continuously — two individually legitimate permissions can combine to produce consequences the user never anticipated. The traditional least-privilege principle can only answer "is the tool authorized" but cannot judge "is it reasonable to invoke it in a specific context." The author thus proposes a long-neglected third pole — the runtime enforcement layer — arguing that only the runtime possesses the full context of Agent behavior and can perform intent-alignment assessment on each tool call. On this basis, the article suggests risk-graded human approval triggers along two dimensions: irreversibility and data exposure surface, and offers a pragmatic power distribution framework: developers define capability boundaries, users declare task intent, and the runtime dynamically judges whether behavior deviates from the objective.