12 AI Agent Frameworks Reviewed: None Can Stop an Agent from Overspending

A Problem All 12 Frameworks Ignore

When your agent can call payment tools, who stops it from draining your account?

Pink Agentic AI Payments ran a systematic review of 12 of the most popular agent frameworks, and the result is blunt: not a single one documents a currency-denominated budget for tool calls.

This is not a minor gap. LangChain/LangGraph, CrewAI, OpenAI Agents SDK, Vercel AI SDK, Google ADK, Microsoft AutoGen, Microsoft Semantic Kernel, LlamaIndex, Pydantic AI, Mastra, Claude Agent SDK, and smolagents — the frameworks developers reach for most when building autonomous agents — can let you pause an agent, cap its steps, or require human approval. But none of them lets you say "this agent can spend at most $50 today" and actually enforce it.

The methodology behind this survey is rigorous: for each framework, only the official documentation site or GitHub source was consulted — no blog posts, third-party tutorials, or AI-generated summaries. Every "yes" or "partial" verdict comes with a URL and a quoted passage of no more than 25 words, so anyone can independently verify it.

What the Eight Review Criteria Reveal

The survey is organized around 8 criteria (C1–C8), checking what each framework offers out of the box:

• C1 Human-in-the-loop approval before tool execution
• C2 Programmable per-tool guards/hooks that can block a call at the code level
• C3 Maximum steps/iterations/turns limits
• C4 Per-run token or LLM cost budgets
• C5 Currency-denominated spending budgets for tools
• C6 Tool call rate limits / max calls per tool
• C7 Idempotency/deduplication for tool calls
• C8 Built-in audit logging of tool calls (including arguments)

The 12 frameworks score 0/12 on C5. That's fair enough — they are general-purpose agent frameworks, not payment products, and currency-denominated spending caps were never their job. But the problem is this: once an agent can actually call payment tools, that gap becomes a real exposure.

MCP (Model Context Protocol) is a protocol open-sourced by Anthropic in late 2024 to standardize how large language models connect to external tools and data sources. Before MCP, every agent framework defined tool interfaces in its own way, which meant the same tool had to be re-adapted for each framework. MCP abstracts tools as "servers," and models call the "tools" these servers expose through a unified JSON-RPC interface — so capabilities like payments, databases, and file systems can be implemented once and reused across frameworks. The spending-control product mentioned in this article is distributed precisely as an MCP server, meaning it can be mounted by any agent framework that supports MCP without per-framework integration. This also explains why payment-layer controls can exist independently of the agent framework — MCP provides a natural, framework-agnostic execution boundary.

What the Frameworks Do — and What They Miss

The data shows frameworks do reasonably well on "general-purpose controls" but are collectively blind on "payment awareness."

The strongest area: step limits. 9 of 12 frameworks document a max-step or max-iteration limit (C3) — the highest coverage of any control in this survey.

Human approval is widely covered but inconsistent in depth. 10/12 frameworks document some form of human-in-the-loop approval (C1 yes or partial), but only some can intercept a specific tool call before it executes. smolagents' agent.interrupt(), for example, stops the whole run rather than approving a single action.

Programmable blocking hooks are also fairly common. 9/12 frameworks document hooks that can block a specific tool call at the code level (C2), rather than relying only on prompt instructions.

Cost budgets are weak. Only 5/12 frameworks document any token- or cost-based run budget (C4), and of those, only 2 are a clean "yes" while 3 are "partial."

Rate limits are almost nonexistent. Only 3/12 frameworks document tool call rate limits or max calls per tool (C6); 8 frameworks have no relevant documentation at all.

Idempotency is the biggest blind spot. Not a single framework (0/12) documents built-in idempotency or deduplication for tool calls (C7), and 9 frameworks have no documentation on this point whatsoever. The implication: a payment call that times out or is retried may execute twice, unless the payment layer handles deduplication itself.

Audit logging is equally missing. No framework documents a built-in audit trail of tool calls with arguments (C8). 6 frameworks have tracing, telemetry, or in-memory run logs that record tool calls, which were classified as partial.

One combination stands out: 8/12 frameworks offer both an approval step and a step limit, yet still no currency-denominated budget. That points to a core issue — step limits and approval prompts don't know how much money each step costs.

Idempotency is a core concept in distributed systems: performing the same operation once or many times produces the same result. In payments, this means that even if a request is retried due to a timeout, funds should only be deducted once. The industry-standard approach is for the client to generate a unique idempotency key for each payment, and for the server to record keys it has already processed — returning the original result for duplicate keys instead of re-executing. Agent frameworks are naturally prone to retries: network timeouts, model re-planning, and step loops can all cause the same tool call to be issued multiple times. If the framework doesn't include idempotent deduplication, developers must maintain idempotency keys and dedup tables in the payment tool implementation themselves — otherwise a payment that should have failed could turn into two successful charges.

Why General-Purpose Controls Aren't Enough

Writing spending rules into the system prompt ("never spend more than $100") is not a control. A prompt is a suggestion to the model, not an enforcement point — nothing stops a tool call that ignores it.

A step limit halts the agent after N actions, regardless of whether those actions moved $5 or $50,000. An approval prompt says "approve this tool call" without telling the approver the amount, the payee, or whether it complies with policy — unless the developer builds that logic themselves.

The place these checks actually need to live is the point where payment executes. If an agent can call a tool that moves money, then budgets, payee allowlists, and approval steps must exist in the payment layer — not in the agent framework's generic step or token counter.

The retry scenario matters just as much. Since none of the 12 frameworks documents built-in idempotency, a payment call that times out or is retried may execute twice, unless the payment layer handles deduplication itself.

The limitations of system prompts as a control deserve elaboration: a system prompt is part of the model's reasoning context. The model "considers" it when generating its next action, but no runtime mechanism enforces compliance. This is fundamentally different from a code-level guard — a guard intercepts before a tool call actually executes and throws an exception, making it a deterministic enforcement point. Putting "don't spend more than $100" in a prompt is effectively outsourcing financial risk control to a soft constraint that can be overridden by later conversation content, bypassed by a jailbreak prompt, or simply ignored due to model hallucination. Truly reliable budget control must happen on the code path where the tool executes: before the payment tool is called, check cumulative amount, payee, and per-transaction limits — and if any check fails, refuse to execute and return a structured error, rather than hoping the model will comply on its own.

The Product Logic Behind the Gap

This survey was published by Pink Agentic AI Payments — the company builds exactly the kind of per-agent spending control product distributed as an MCP server, including budget setting, amount rules, payee allow/block lists, and human approval above thresholds, all checked before payment executes. The survey findings are directly relevant to their product, and the original article discloses this.

That doesn't make the data untrustworthy. On the contrary: the complete dataset, methodology, and a compute.py script that regenerates every number from survey.csv are all on GitHub under a CC BY 4.0 license. If you maintain any of the frameworks listed, and you find a cell that's wrong or outdated, you can file a correction issue with a documentation link.

Transparency is high enough that the data speaks for itself. For developers building agents that can call payment tools, this survey points to one clear fact: generic guardrails in agent frameworks are no substitute for purpose-built controls in the payment layer. If your agent can spend money, you need to solve budgeting, allowlists, approvals, and idempotency separately — outside the framework.

One-line Summary

A survey shows none of 12 mainstream agent frameworks offers currency-denominated spending budgets, and generic guardrails are no substitute for payment-layer controls.

Paragraph Summary

Pink Agentic AI Payments conducted a systematic documentation review of 12 mainstream agent frameworks — including LangChain, CrewAI, and OpenAI Agents SDK — evaluating out-of-the-box tool-call controls across 8 criteria. The results: currency-denominated spending budgets (C5) score 0/12, idempotency (C7) and argument-level audit logging (C8) also score 0/12, while only step limits (9/12) and human approval (10/12) have decent coverage. 8 frameworks offer both approval and step limits yet still no currency budget, showing that generic controls don't know how much each step costs. The article argues that system prompts aren't enforcement points, step limits can't map to amounts, and the lack of idempotency can cause duplicate charges on retries. The conclusion: when agents can call payment tools, budgets, allowlists, approvals, and idempotency must be solved at the payment execution layer — not left to the framework's generic guardrails.