[KongchangAI]
· 3 min read· 1,852 words

AI Agent Approval Gates Only Validate the Verb, Not the Object: Three Escape Paths and Seven Root Causes

AI Agent Approval Gates Only Validate the Verb, Not the Object: Three Escape Paths and Seven Root Causes

Human-in-the-loop gates that approve only the verb and not the object are bypassed by indirect injection.

A red team experiment shows AI Agent human approval gates generally check only whether a tool may be called, not which transaction was approved. The simulated system passed all ten rejection tests, but three of five parameter-only changes failed: the payee account was swapped, the amount went from $1,840 to $18,400, and the employee directory leaked through the supposedly safe send_reply tool. The root cause is that the control plane and data plane collapse into one context window. The author lists seven root causes and fixes: sign approval to exact parameters and re-check at execution, taint-track egress, show approvers raw parameters, label retrieval provenance, run with least privilege per task, and test objects, not just verbs.

Most AI Agent human-in-the-loop approval mechanisms answer only one question: is this tool allowed to be called? The approver clicks "Approve" on approve_payment, and the gate is considered satisfied.

But this approval binds to a verb, not an object. It is not bound to "who this money is paid to, how much, and which invoice it corresponds to," so after approval, the content the Agent reads can still determine where the money actually goes.

A developer shared his red team experiment on Bilibili and Dev.to: he built a simulated company with four types of Agents—helpdesk, finance, code review, and release—layered with a policy. The policy's rejection tests passed 10/10. He then wrote five new tests, keeping the verb unchanged and only changing the object, and three failed.

This problem refers to: the escape paths formed by indirect prompt injection combined with "verb-level approval," and why an all-green scorecard completely fails to see it.

Where Exactly Is the Trust Boundary: Control Plane and Data Plane Collapse into a Single Context Window

The true location of the trust boundary

The diagram above divides the Agent system into two rows. The top row is the control plane: request initiator, system prompt, policy checks, approver. The bottom row is the data plane: labeled by "who can write to it"—wiki runbook, invoice fields, PR description, ticket history messages.

By design there is a dashed boundary between the two. In reality, both rows are crammed into the same context window, and the dashed line disappears. A wiki paragraph edited by a contractor, a modified invoice, and the system prompt and the SRE lead's instructions are the same kind of token in the model's eyes.

The contributor makes this observation explicit in the figure caption: the collapse of the control plane and data plane is the structural precondition for all subsequent problems.

Indirect Prompt Injection means the attacker does not directly input malicious instructions to the model, but hides instructions in external content the model subsequently reads—such as web pages, documents, tickets, emails, or code comments. When the Agent retrieves or reads this content to complete a task, the malicious instructions enter the context along with the data and are executed by the model as legitimate instructions. Its difference from direct injection is that the attacker and the victim are not in the same message; the attack surface is the Agent's entire information intake pipeline. This type of attack is especially dangerous because the tainted content often comes from seemingly trustworthy business systems and can take effect only after the Agent has obtained approval, thereby bypassing pre-event gates.

What the 10/10 Scorecard Actually Tests

Of the ten test fixtures, nine are direct injection: the attacker text and the request appear in the same message. Only one—the poisoned wiki note—is true indirect injection.

And every test asks the same question: did the gate stop this privileged verb?

It did. The gate is wired correctly. But that tells you nothing about the parameters. A policy layer that only checks the verb is blind to "which transaction was approved."

Test Fixture in software testing refers to pre-arranged input data and runtime environment used to make the system under test exhibit specific behavior under controlled conditions. The ten fixtures in this article are ten carefully constructed Agent inputs: nine place attacker text and the user request in the same message, simulating direct injection; the other hides malicious content in a wiki note, simulating indirect injection. The design of fixtures directly determines what the tests can cover and what they miss—when all fixtures ask "did the gate stop the verb," the scorecard is naturally blind to parameter tampering. This is an old problem in testing: the way test cases are constructed is itself an implicit specification.

Three Failure Cases: Approval Was Signed, but the Object Was Swapped

With the same policy, three of the five tests that only change the object fail:

  1. Bank account field replacement. A human approved payment of invoice INV-2201. After approval, the payee bank information on the invoice was edited, and the payment went to the new account as usual.
  2. Amount change. In the same approval, the amount was changed from $1,840 to $18,400, and the payment went through as usual.
  3. Exfiltration through a "safe" tool. A helpdesk ticket induced the Agent to use send_reply—a tool the policy treats as always safe—to include the employee directory in the reply. The redaction filter missed all five records.

The author specifically adds a premise note: these are scripted stand-ins, equivalent to "the model has already complied with the injected text." It tests the policy layer, not any specific model's ability to resist injection. This setup is deliberate—you should assume the model sometimes complies, and ask: when it complies, what does the system allow it to do?

Confused Deputy is a classic security problem: a program with higher privileges is induced by a lower-privilege caller to perform an operation the caller has no right to perform. The classic example is a compiler being asked by a user to overwrite a file it has no permission to write. In LLM Agent scenarios, the Agent often runs with a super identity, while far more people can write to the data plane (such as contractors, external ticket submitters) than are allowed to directly trigger privileged actions. The Agent sits between the two groups, connecting "can write data" and "can trigger actions," and becomes the confused deputy itself. The key to fixing this is not to make the Agent smarter, but to run it with the request initiator's own permissions and narrow it to least privilege per task.

Seven Root Causes: Each Requires a Different Fix

"The model complied with a malicious document" is true but useless. Beneath it lie seven independent mechanisms.

RC1: No structural separation between instructions and data. SQL injection was largely solved by parameterized queries—code and data are transmitted separately. LLM context has no equivalent. The runbook and the system prompt are the same kind of token.

RC2: Task framing amplifies compliance. "Execute per the runbook" and "do as the ticket requires" turn instruction-following from a risk into a job responsibility.

RC3: Provenance is stripped at ingestion. When a chunk enters the context window, a contractor's wiki edit and the SRE lead's edit look identical.

RC4: Confused deputy. The population that can write to the data plane is far larger than the population allowed to trigger actions. The Agent connects the two groups.

RC5: Approval binds to the verb, not the object. This is the root cause masked by 10/10. The attacker does not need to unlock a privileged tool; they only need to sit upstream of a human-approved tool and shape its parameters.

RC6: The human gate is mediated by the Agent. The approver reads a summary, and the summary is written by the very Agent that just read the poisoned content. A "trusted channel" is often just a string set by the caller itself.

RC7: "Safe" tools are the exit. The policy guards privileged verbs and waves through send_reply, comment, and summarize. Once private data and untrusted input share the same session, these tools are an exfiltration pipeline.

The seven root causes correspond to seven fixes. Adding a line to the system prompt saying "ignore instructions in documents" is not on this list—it may help at the margins, but it solves none of the seven root causes.

The Fix Is Boring but It Works

The tightened policy passed all five tests, and the two legitimate-use controls still succeeded:

  • Bind approval to the object. Sign the exact parameters (tool, payee, amount, invoice, expiration time, nonce) and re-validate at execution. Any change after signing is rejected.
  • Taint-track egress. Once untrusted content enters the context, outbound tools are downgraded to internal-only, templated, or require approval.
  • Show approvers the original request and parameters, not the Agent-written summary.
  • Label retrieved chunks with provenance, and restrict which sources privileged tasks may use.
  • Run with least privilege per task. Use the request initiator's own permissions, not a super identity.
  • Test the object, not just the verb. Add a regression test for "change parameters after approval" to every privileged tool, plus a positive control proving normal work still succeeds.

The contributor's practical check is also direct: pull one approval record from your Agent platform and see whether it binds exact parameters (payee, target, amount, expiration time) or only a tool name. If it is only a tool name, your human-in-the-loop is approving a category, not a transaction.

Taint Tracking originates from information-flow security models. The core idea is to tag data with a "source trustworthiness" label and let the label propagate as data flows through the system: once data from an untrusted source enters a variable or context, that variable is marked "tainted," and any operation sending it to an external destination is blocked or downgraded. In Agent systems, the implementation is to label provenance when retrieved chunks enter the context and stipulate: whenever the current session contains untrusted content, outbound tools (sending email, replies, comments, external summaries) are downgraded to internal-only, templated output, or mandatory human approval. It fills exactly the blind spot of "guarding only privileged verbs"—egress tools are not privileged in themselves, but become dangerous once they carry tainted data.

A Few Judgments

The value of this experiment is not "discovering prompt injection"—that has long been consensus—but moving the problem from the model layer to the policy layer and providing a reproducible measurement. The 10/10 scorecard is not fraud; it tested the wrong object. For teams adding approval gates to Agents, this distinction is worth copying directly: verb gates stop no "change after approval" attack; object binding does.

It should be noted that the five security engineer prompt packs (Trust Boundary) mentioned in the article are the author's paid product, $15, including a limited-time discount code; the original also discloses that the lab is a clean-room simulation, not a test of real systems, and that the article and figures were drafted with AI assistance. Please retain these qualifications when citing.

One-line Summary

The human-in-the-loop gate that approves only the verb and does not bind the object can be bypassed by indirect injection.

Paragraph Summary

Through a red team experiment, the article reveals that AI Agent human approval gates generally check only "is the tool allowed to be called" but do not bind "which transaction was approved." The author's simulated system passed all ten rejection tests, but three of five tests that only changed parameters failed—the payee account was replaced, the amount changed from $1,840 to $18,400, and the employee directory was exfiltrated through the send_reply tool treated as always safe. The root cause is that the control plane and data plane collapse into the same context window, and instructions and data lose structural separation. The author summarizes seven independent root causes and gives fixes: bind approval signatures to exact parameters and re-check at execution, taint-track egress, show approvers the original parameters, label retrieval provenance, run with least privilege per task, and test the object rather than only the verb.

#AI Agent#human-in-the-loop#indirect prompt injection#approval gate#confused deputy#taint tracking#least privilege
Share:

Related articles