The AI Agent Payment Problem: Why LLMs Shouldn't Hold API Keys Directly

The Agent Execution Wall: Autonomy Meets Irreversible Transactions

Having AI agents plan, search, compare prices, and draft contracts is now fairly mature capability. But when an agent needs to actually execute an irreversible transaction — paying a deposit, confirming a booking, paying a vendor, releasing milestone funds — production-grade architectures tend to fall apart at exactly that moment. A team building a multi-agent coordination workflow on Reddit coined this bottleneck the "Agent Execution Wall" and shared their approach to solving it.

The core problem is straightforward: planning and reasoning can tolerate probabilistic outputs, but financial and legal execution must be 100% deterministic. Mixing both in the same runtime amplifies risk exponentially.

Three Layers of Risk: Why You Can't Hand Keys to an LLM

The original post breaks down the risks of this type of architecture into three layers, each sufficient on its own to make the approach untenable in an enterprise environment.

The Prompt Injection Nightmare

Once an agent holds payment API keys, virtual cards, or signing credentials, any indirect prompt injection or logical infinite loop means an unbounded "financial blast radius." An attacker only needs to embed malicious instructions in some external content the agent reads — and it could be tricked into initiating a real transfer. Traditional sandboxing approaches try to "contain" credentials inside the agent runtime, but as long as credentials are callable within that runtime, the attack surface never disappears.

What is Indirect Prompt Injection? Unlike direct prompt injection — where a user manually types malicious instructions into a chat box — indirect injection hides malicious instructions in external content the agent actively reads: web pages, PDF documents, email bodies, database query results, even vendor-returned quotes. When the agent "reads" this content and passes it into the LLM context, instructions embedded within it get executed by the model as if they were legitimate intent. This is particularly dangerous for agents with payment permissions because the attack surface is extremely diffuse — any external data source the agent touches could serve as a carrier. Defenders can't enumerate every injection point, while attackers only need to succeed once to trigger a real transfer.

The Absence of Determinism

LLMs are probabilistic by design. They excel at generating, inferring, and completing — but they cannot guarantee strictly consistent outputs every time. Financial and legal execution, by contrast, demands absolute determinism: the same inputs must produce the same, auditable results. Handing execution authority to a system that fundamentally "guesses" is simply not defensible from a compliance standpoint.

The Enterprise Compliance Red Line

Risk officers and compliance teams will never approve an autonomous model holding unrestricted signing authority. This isn't a matter of technical preference — it's a hard regulatory and audit constraint. Any architecture that tries to work around this will struggle to pass enterprise procurement and security reviews.

The Architectural Solution: Propose, Don't Execute

The team's solution doesn't try to make credentials "safer" inside an agent. Instead, it enforces a clean separation of responsibilities at the protocol layer, with one clear principle:

Agents propose. Humans approve. Infrastructure executes.

They published an open-source Model Context Protocol (MCP) server — io.github.covaltpay/covalt-gateway — that exposes only "propose-only" tools to the agent:

• calculate_fees: Calculates transaction economics and profitability before any proposal;
• get_trust_score: Validates the counterparty's risk profile;
• propose_pact: Assembles a structured, conditional commitment (called a Pact) across Stripe and Wise channels.

The critical point: the agent never touches card numbers, banking APIs, or any fund movement operations at any point.

Human Consent as the Only Execution Switch

When the agent calls propose_pact, the platform only records the "intent" and returns a cryptographically signed human approval link. The user confirms intent on their own trusted device — only then does the platform actually orchestrate execution.

The elegance of this design lies in zeroing out the blast radius. If the agent hallucinates or gets poisoned, the worst possible outcome is a rejected, unexecuted proposal — which is nothing more than inert data, causing £0.00 in damage. In other words, even if an attacker fully controls the agent's reasoning process, they cannot bypass this deterministic human checkpoint.

What is MCP (Model Context Protocol)? MCP is an open standard proposed by Anthropic in late 2024 to govern how AI models interact with external tools and data sources. Its core idea is to separate "what a tool can do" from "what a tool actually executes" — the server declares tool input/output schemas, the client (the LLM runtime) calls them via protocol, and the entire process can be intercepted and audited by an intermediary layer. MCP has since gained support from OpenAI, Google DeepMind, and other major vendors, and has established a global tool registry. For the scenario in this article, MCP's value lies in providing a standardized "expose only partial capabilities" access layer — the server can precisely control which tools are visible to the agent, restricting agent permissions at the protocol layer rather than the runtime layer. This is far more reliable than making permission decisions inside the agent itself.

Is This Over-Engineering?

It's worth discussing: the "propose-approve" model inherently trades some degree of automation for safety and compliance. For high-frequency, low-value, low-risk scenarios, requiring a human to click an approval link for every transaction may not be ideal from a UX standpoint. The get_trust_score and calculate_fees tools in the original post are actually doing pre-work for the approval step — making human decisions as lightweight as possible.

From an industry perspective, this aligns with the broader "Human-in-the-Loop" philosophy of agent safety. The real debate isn't whether humans should be involved, but at what granularity — approving every transaction individually, or setting thresholds and rules for batch authorization. The approach described here is conservative, routing all irreversible transactions through human confirmation first. For high-risk domains like payments, this is a reasonable default.

What is Human-in-the-Loop (HITL)? HITL is a design paradigm in machine learning and automation systems that mandates human judgment at critical points in an automated process, rather than letting the system run fully autonomously. In the agent safety context, HITL doesn't simply mean "manually review every step" — it's an engineering trade-off between risk and efficiency. By setting confidence thresholds, amount limits, and operation type whitelists, a system can remain fully automated for low-risk operations and only trigger human confirmation when it exceeds predefined boundaries. This contrasts with the "full interception" strategy in this article — routing all irreversible transactions through human approval — which is a conservative default well-suited for initial deployments or highly regulated industries. Over time, conditional rules can be introduced to expand automation coverage while maintaining human control over high-risk operations.

Takeaways for Developers

For teams building autonomous agents, this post raises a question that demands an answer: Where exactly does your system draw the line between reasoning/planning and high-risk irreversible execution?

Several principles worth borrowing directly:

• Never let credentials enter the LLM's callable scope, even with a sandbox;
• Completely decouple "intent" from "execution" at the protocol layer — intent can be probabilistic, execution must be deterministic;
• Use cryptographically signed human approval as the final gate for irreversible operations;
• Expose "read-only / propose-only" tools via standard protocols like MCP to reduce integration overhead while narrowing the attack surface.

The project is registered in the MCP global registry and offers a developer playground, video demo (covaltpay.com/developers/mcp), and open-source repository (github.com/covaltpay/Covaltpay) for teams who want to try the tool schema directly. As agent-driven payments become an increasingly real requirement, holding the execution boundary is an engineering problem that can't be avoided for the foreseeable future.