AI Agent Security: The Attack Surfaces Teams Overlook Most

Most AI security discussions focus on prompt injection. It is indeed one door into an agent, but it is often not the most critical one.

A jailbreak checklist tests what an agent can be persuaded to do. The other three questions test what happens after it has been persuaded — and these surface far more problems. This is exactly the attack surface teams overlook most.

The Overlooked Attack Surface: Prompt Injection Is Just the Tip of the Iceberg

Current industry attention on AI agent security is overly concentrated on prompt injection and jailbreak attacks. Security teams pour enormous effort into building jailbreak checklists, trying to enumerate every kind of cleverly crafted prompt to verify whether a model can be induced to output harmful content.

But this line of thinking has a fundamental blind spot. A jailbreak checklist measures "what an agent can be persuaded to do," while the real danger lies in "what it can reach once it has been persuaded." The former is an input-side problem; the latter is a problem of permissions, memory, and trust chains — and the latter is usually what causes actual damage.

The framework proposed in this article comes from the security assessment firm stonepathit. Its core argument: don't just ask "can it be fooled," but assume it will definitely be fooled, then dig into three deeper questions.

Question One: What Is It Allowed to Do?

The first question is not "can the agent be deceived," but "what can a deceived agent reach."

Consider an agent with a shell tool. It runs on top of the process that hosts it, and the user's operating system permissions don't automatically come with it. If the model "loses its mind" due to a bad prompt, a poisoned document, or a pure mistake, the blast radius is everything that process can touch.

This shift in perspective is crucial. Traditional security thinking asks "can it be deceived," but a more effective approach is to assume deception will inevitably occur, then assess the resources and system boundaries the agent can reach once deceived.

If an agent with shell access runs under excessive privileges, a single successful prompt injection could lead to data leakage, system destruction, or even lateral movement. The principle of least privilege applies just as much in the age of AI agents — even more so.

Question Two: What Does It Remember?

Memory makes agents useful, but it also makes injection persistent.

A note written in one session, read back in the next, is essentially input from the past. If there is no mechanism to check the note when it is read back, then whoever wrote that note last week is still manipulating the agent today. The original conversation is long gone, and the opportunity to catch the problem at the time has already been lost.

This is a severely underestimated risk vector. Traditional web application security models assume each request is independent, or at least that session boundaries are clear. But agents with long-term memory break that assumption — yesterday's malicious input can become today's hidden instruction.

The insidiousness of memory poisoning lies in this: it doesn't happen in a live conversation, so it doesn't trigger any real-time security alert. A seemingly harmless note can carry an injection payload that isn't triggered until days or even weeks later.

Question Three: What Does It Trust?

When agents hand off work to one another, or when the output of one step becomes the input of the next, every hop is an unverified trust gap.

Pipelines often validate at the edge — where user input arrives — and then trust everything internally. But a model's output is untrusted text, no matter which model it came from. Every handoff is an input boundary, whether or not anyone explicitly drew that line.

The popularity of multi-agent systems makes this problem even more prominent. When Agent A calls Agent B, which then calls tool C, the trust assumptions in the intermediate stages are often never examined. Every hop can become a channel for an attacker to inject instructions.

The secure approach is to treat every agent-to-agent handoff as untrusted input, just like user input coming from the internet.

Why These Three Questions Are More Effective Than a Jailbreak Checklist

These three questions surface more problems than a jailbreak checklist because they cover a more complete attack lifecycle.

A jailbreak checklist only covers the "entrance" — a single stage — while the questions of permissions, memory, and trust chains run through the agent's entire operational lifecycle. More critically, these three questions are exactly what a buyer's security team will ask — rather than waiting for someone else to challenge your agent security, answer them yourself first.

From a practical standpoint, these three questions provide a structured security assessment framework:

• Permissions layer: What permission context does the agent run in? Has least privilege been applied?
• Memory layer: Is there validation for memory writes and reads? How is persistent data across sessions audited?
• Trust layer: Is there implicit trust in data flows between agents and between steps?

If you'd like your agent security tested by people who specialize in trying to break through these answers, consider booking a scoped assessment.

Recommendations for Developers and Security Teams

Agent security cannot rely solely on prompt-level defenses. When building AI applications, you need to consider permission boundaries, memory security, and trust models at the system architecture level.

Specifically: limit agent runtime privileges to the minimum necessary scope; implement write review and read validation for persistent memory; and perform input validation at every handoff between agents. These measures cannot eliminate risk entirely, but they can significantly reduce the blast radius after an attack.

Ultimately, the security question isn't whether an agent will be deceived — it will. The question is how far it can go once deceived.

Background Notes

Prompt injection refers to an attacker crafting input that causes a model to treat the content as instructions rather than data, thereby bypassing existing constraints. It typically falls into two categories: direct injection, where the user writes malicious instructions directly in the conversation; and indirect injection, where malicious content is hidden in external data the model reads — such as web pages, documents, emails, or tool return results — and the model unknowingly executes it as instructions. Jailbreaking, by contrast, focuses more on bypassing the model's own safety alignment to make it output content it should refuse. What both have in common is that they locate the problem at the "input side," trying to solve it by filtering prompts or strengthening alignment. But once an agent has tool invocation and persistent state, the attack surface is no longer confined to input — it extends to the tools it is allowed to call, the memory it can read and write, and its default trust in external data.

The principle of least privilege is a classic security principle requiring that each subject be granted only the minimum permissions necessary to complete its task, and only for as long as necessary. In the AI agent context, this principle needs to be implemented at multiple levels: operating system user permissions, container or sandbox restrictions, tool allowlists, network egress rules, and fine-grained credential authorization. Many agents, for convenience, are granted excessive privileges — for example, an agent whose only job is organizing documents might have read-write access to the entire file system or a token to access cloud resources. Once that agent is manipulated by indirect prompt injection, the attacker can use its permissions to exfiltrate data or move laterally. Therefore, during assessment you should first map out a "permission map," clarifying which permission context it runs in and what resources it can reach, then judge whether the blast radius after deception is acceptable.

Memory poisoning refers to an attacker writing malicious instructions or false information into an agent's long-term memory, so that it is repeatedly read in subsequent sessions or tasks and executed as trusted context. Compared to real-time prompt injection, it is harder to detect because the contamination occurs in the past, the trigger point may be days or even weeks later, and it does not appear in the current conversation record. Common write channels include user conversation summaries, tool return results, scraped web content, and notes automatically generated by the agent. Defensive approaches typically include: tagging the source and grading the trustworthiness of content written to memory; re-validating on read rather than trusting by default; isolating content from external untrusted sources in separate storage to avoid mixing it with system instructions or user preferences; and setting up memory expiration and manual review mechanisms. The core idea is to treat memory as untrusted data, not as trusted instructions.

A multi-agent system refers to an architecture in which multiple agents collaborate to complete tasks, with common patterns including supervisor-worker, pipeline, and debate. Such architectures improve the ability to handle complex tasks but also introduce new trust boundaries: Agent A's output becomes Agent B's input, and B often assumes by default that A's content is trusted instructions. An attacker only needs to contaminate one link — for example, having a web page A scrapes contain hidden instructions — to leverage A's "trusted identity" to influence B's behavior. The secure approach is to treat every agent-to-agent handoff as an input boundary, tag the source of transmitted content, apply structural constraints and necessary validation, and restrict downstream agents to executing only allowlisted actions. In other words, you cannot lower your scrutiny just because the content comes from "one of your own."