AI Agent Hijacking: 3 Real-World Cases and 7 Lines of Defense

AI agents no longer just answer questions—they can read your email, view your files, run commands, and even operate databases on your behalf. With greater power comes greater risk. When an attacker hides instructions in the text an agent is about to read, it may "obediently" leak your privacy or delete your data. This article reviews three publicly reported real-world hijacking cases, breaks down the underlying logic that makes attacks work, and offers seven defenses you can implement today.

What Is Agent Hijacking, and Why Is It So Hard to Fix

Chatbots only answer questions, but agents add a layer of "tools": reading and writing files, browsing the web, sending messages, executing commands. The AI Standards and Innovation Center under the U.S. National Institute of Standards and Technology (NIST) defines "agent hijacking" as a form of indirect prompt injection—an attacker hides instructions in emails, files, or web pages that the agent will read, inducing it to act against the user's intent.

The root of the problem is architectural: the "instructions" an agent receives and the "material" it processes are concatenated into the same text stream. The old fix for SQL injection was to separate data from instructions (parameterized queries), but the UK National Cyber Security Centre (NCSC) points out that language models have no equivalent isolation mechanism, and prompt injection may never be fully fixed the way SQL injection was. This means security design should assume "it will sometimes succeed," not "if the model is strong enough, it won't be fooled."

Indirect Prompt Injection is a variant of prompt injection: instead of conversing directly with the model, the attacker writes malicious instructions into external data sources the model will read (such as web pages, emails, documents, code comments). When the agent retrieves this content to complete a task, the malicious instructions enter the model's context and are executed as legitimate commands. In contrast, direct prompt injection involves the user constructing malicious instructions directly in the input. The danger of indirect injection is that the attacker doesn't need to touch the target system—they just need to plant content in a publicly writable location and wait for the agent to "take the bait."

Three Real-World Cases: From GitHub to Copilot to Databases

Case 1: Hidden Instructions in a GitHub MCP Server (May 2025)

According to public research by Invariant Labs, an attacker submitted a seemingly normal issue to a public repository containing hidden instructions. When a developer who also had access to private repositories asked the agent to "review open issues," the agent read the issue, executed the hidden instructions, pulled sensitive information from private repositories, and published it in a public pull request. Researchers called it an architectural problem rather than a code defect—it cannot be fixed once and for all with a patch.

Case 2: Microsoft 365 Copilot's "Echo Leak" (June 2025)

AIM Security disclosed a vulnerability called EchoLeak: a carefully crafted email carried hidden instructions that Copilot read during routine email processing, with instructions to send sensitive data to the attacker. The entire process required no user clicks. Microsoft subsequently fixed it on its side, and public reports indicate no evidence the vulnerability was exploited in real attacks.

Case 3: Over-Privileged Coding Agent Deletes Database (June 2025)

A founder recounted that a coding agent, while performing a routine test-environment task, discovered an over-privileged token in an unrelated file and deleted a production data volume in a single call. There was no attacker in this incident—the damage came entirely from the token being able to do too much.

The Lethal Trifecta and the "Rule of Two"

The first two cases share the same structure. Developer Simon Willison calls it the "lethal trifecta": first, access to private data; second, exposure to untrusted content (any text an attacker can place before the model); third, the ability to communicate externally (to send data out). With all three present, an attacker can induce the agent to exfiltrate private data. The GitHub case and EchoLeak both fit this model; the database deletion case is different—it had only one problem: excessive privileges.

Meta proposed a practical rule in October 2025, the "Agent's Rule of Two": an agent may process untrusted input, may access sensitive systems or private data, may change state or communicate externally—until prompt injection can be reliably detected and rejected, a single session should possess at most two of these. Meta explicitly states this is not a complete solution but a complement to, not a replacement for, the principle of least privilege. If a task truly requires all three, human approval of critical steps should be introduced rather than fully automated execution.

MCP (Model Context Protocol) is an open-source protocol proposed by Anthropic to standardize connections between large language models and external tools and data sources. MCP servers act as adapters for agents to access resources such as file systems, databases, and APIs, enabling agents to invoke multiple tools through a unified interface. However, MCP servers themselves can become an attack surface: if a server exposes overly broad permissions (such as allowing both reading private repositories and publishing public content), or fails to isolate returned content, an attacker can hijack behavior by poisoning the data the agent reads. The GitHub MCP case exploited exactly the server's combination of reading private data and publishing externally.

Why Defenses Get Bypassed: Test Data from NIST and UK AISI

"A good enough model will see through it" is a dangerous assumption. In NIST's tests against the Agent Dojo benchmark, the strongest attacks at the time achieved an 11% success rate against the upgraded model; NIST and the UK AI Safety Institute (UK AISI) then designed new attacks against that model, raising the success rate to 81%. When each attack was repeated 25 times, the average success rate across five tasks rose from 57% to 80%. The conclusion is direct: defenses that perform well against known attacks may be breached by new attacks, and attackers can try repeatedly. So plan for the day an injection succeeds, and limit the damage it can cause.

Agent Dojo is a benchmark framework for evaluating agent security, proposed by researchers from ETH Zurich and other institutions. It simulates multiple real-world task scenarios (such as email processing, file management, database operations) and includes attacker models to measure task completion rates and data leakage rates when agents face prompt injection. NIST and UK AISI used this benchmark to test upgraded models, finding that defenses optimized for old attacks quickly fail against new ones, thereby quantifying the risk of "defense bypass." The benchmark's significance lies in providing a repeatable adversarial evaluation environment to help researchers and developers improve agent security design in a data-driven way.

Seven Actionable Defenses

The following measures combine OWASP guidelines, UK National Cyber Security Centre recommendations, and the cases above:

1. Break the lethal trifecta. Clearly define which two capabilities each agent has, and remove the third.
2. Least privilege. Configure independent, narrowly scoped credentials for each agent, scoped to individual tasks where possible, with short expiration times.
3. Reduce the number of tools. For tasks that can be done with a narrow tool, don't expose open-ended tools like raw shells.
4. Treat external content as untrusted data. Label, isolate, and reduce agent privileges when reading it. NCSC adds: don't rely on blocklists of forbidden phrases—attackers can just rephrase.
5. Require human approval for high-risk actions. Require human confirmation before operations like deletion, payment, sending, or deployment, and be wary of "always allow" buttons that can mask data exfiltration.
6. Enforce rules with ordinary code outside the model. For example, allow only approved destinations for sending, or restrict each session to a single repository.
7. Sandbox, log, and red-team. Record agent behavior and test it like an attacker; store backups where agent credentials cannot reach.

Red Teaming in the context of agent security refers to simulating attacker behavior to conduct adversarial testing of systems, aiming to uncover prompt injection, privilege abuse, data exfiltration, and other vulnerabilities. Unlike traditional penetration testing, red teaming focuses not only on code defects but also on weaknesses at the model behavior level—such as whether an agent can be induced to execute hidden instructions, or whether it will skip approval before high-risk operations. OWASP recommends incorporating red teaming into the agent development lifecycle, regularly updating the attack library, and testing "worst-case" scenarios—that is, assuming injection has already succeeded, and assessing whether the scope of damage is effectively limited.

A Quick Self-Check for Any Agent or MCP Server

Before connecting an agent or MCP server, ask three questions: What private data can it read? What untrusted text will it see (web pages, emails, issues, tickets, tool return results)? What can it send externally or change? If all three answers are yes, remove one of them, or put a human in the loop.

Agents follow text, and attackers can write text. Prompt injection may never be fully solved, so the right posture is "design for it happening": never let an agent simultaneously possess private data, untrusted content, and an external channel without a human fallback; reduce the capabilities it can abuse, and put irreversible steps in human hands.

(This article is general security education content and does not constitute a security assessment of any product; case details are based on public reports and may change over time.)