AI Agent Tool Definitions Account for 32% of the Bill: The Overlooked Token Cost

AI Agent tool definitions consume 32% of the bill, yet they're often overlooked. Let's break down the Token cost.

Every time an AI Agent calls a tool, the model doesn't "remember" the tool list. Due to the stateless design of the API, the complete tool Schema must be retransmitted with every request. One developer's measurements show: 18 tool definitions consume 2,388 Tokens, which in a 10-step Agent run can account for 36% of input Tokens and 32% of the total bill.

How Does the Model "Know" the Tools

To understand where this cost comes from, we first need to clarify a few basic concepts. Token is the basic unit of API billing, roughly equivalent to 3/4 of an English word. APIs bill per million Tokens, with different prices for input (what you send) and output (what the model generates). Tool Calling is an API feature: you describe available functions, and the model replies "call read_file with these parameters" rather than plain text.

Each function is described via a JSON Schema—a JSON object containing a name, description, and parameter types. Take the edit_file tool as an example: just one tool contains four parameters—path, old_string, new_string, replace_all—each with its own type and description text.

The key point: the API is stateless, retaining no memory between requests. The entire tools array must be sent with every request, alongside the system prompt and conversation history. And an Agent loop—where the model calls a tool, gets the result, and decides the next step—constitutes one request per step. Ten steps means ten copies of the tool list.

Real Measurements: The Actual Token Overhead of 18 Tools

To quantify this problem, the author built a typical coding Agent tool set with 18 tools, covering read_file, write_file, edit_file, list_directory, search_code, find_files, run_shell, run_tests, three git tools, web_search, fetch_url, http_request, query_database, create_issue, send_message, ask_user. The description text is kept at a normal level—one to two sentences per tool.

Using the tiktoken library with the o200k_base encoding (the tokenizer behind current GPT models), the count is: 2,388 Tokens, averaging 133 Tokens per tool. The smallest, git_status, has no parameters and only 54 Tokens; the largest, search_code, has five parameters and reaches 193 Tokens. And this is just the overhead of a single request—the user hasn't even typed the first word yet.

tiktoken and the tokenizer determine where these Token numbers actually come from. Models don't process text by characters or words; they first slice input into Tokens—subword units between characters and words. tiktoken is OpenAI's open-source tokenization library that converts any text into a Token sequence and gives a count, allowing you to estimate cost before calling the API. Different tokenizers correspond to different segmentation rules: o200k_base is the encoding used by newer GPT series, with a larger vocabulary, and its segmentation efficiency for structured text like code and JSON is usually better than the older cl100k_base. The reason to specify the encoding version is that the same JSON may differ by a few percent under different tokenizers, so you must use a consistent standard when comparing costs across models. Counting locally with tiktoken is the most direct way to verify "how much did this request actually cost," and it's also the tool the article recommends readers use to quantify their own costs.

Cost Breakdown of a Ten-Step Agent Run

The author sets up a moderate Agent run scenario: a 500-Token system prompt, a 200-Token task description, 10 steps, each step adding approximately 780 Tokens of history (one tool call and its result), producing 150 Tokens of output.

Tool set	Tool Tokens per run	Share of input	Cost per run	Monthly Schema cost*
All 18 tools	23,880	36%	$0.150	$1,433
The 6 tools needed for the task	9,380	18%	$0.121	$563
No tools (baseline)	0	0%	$0.102	$0

*Based on 1,000 runs per day, 30 days, input pricing only.

With the full tool set, the Schema cost is $0.048/run, accounting for 32% of the $0.150 total cost. Within a single request, the weight of tool definitions is more than three times the sum of the system prompt and task description. At 1,000 runs per day, that's 716 million Tokens per month spent repeatedly describing send_message to a model that's fixing a typo.

Three Practical Strategies for Reducing Costs

Streamline the tool set. Give the Agent only the tools the task truly needs. A code-fixing task doesn't need create_issue, send_message, or query_database. After reducing to 6 relevant tools, the monthly Schema cost drops from $1,433 to $563—saving $870 without modifying any prompt.

Compress the description text. Keep all tools but reduce each description to its first sentence, and remove the description of each parameter: Tokens drop from 2,388 to 1,545, a 35% reduction. However, you need to test whether the model can still select tools correctly—the descriptions are exactly what the model uses to make decisions.

Use a script to quantify your own costs. Run the tiktoken counting script, spend a minute, and you can turn "it's just some JSON" into concrete numbers.

Additionally, the Token unit price is also one of the multipliers. The author mentions that their platform resells GPT-5.6 at $1.70/million input Tokens (official price $2.00), and the same 716 million Token Schema cost drops from $1,433 to $1,218. But saving 15% is far less effective than cutting waste at the source.

Blind Spots This Analysis Doesn't Cover

The author's counting is approximate. Vendors don't send your JSON to the model as-is—they rewrite it into an internal format and add their own instructions, so the real numbers will differ and vary by vendor. It's recommended to verify by comparing the usage.prompt_tokens field in actual calls with and without tools.

Prompt caching will significantly change the cost structure: the tool list sits at the beginning of each request, exactly where the cache discount covers, so with caching enabled the 32% share will shrink. But the author can't show the exact reduction—their usage records don't yet separate cached Tokens. Moreover, switching tool sets at each step breaks the cache, so there's tension between "reducing the number of tools" and "caching the tool list," requiring a trade-off.

The core conclusion is simple: before optimizing anything else, count your tools. They are the part you write once in the prompt but pay for forever.

Prompt Caching is the mechanism mentioned here but not elaborated on. Most vendors cache the prefix of a request—that is, the portion from the beginning that is identical across multiple requests. The cached portion is billed at a significantly discounted price (commonly 10% of the original price), and the uncached portion at the original price. Since the tool list is usually right next to the system prompt at the very front of the request, it's naturally the ideal target for caching: as long as the prefix bytes are completely identical, every subsequent step can be sent repeatedly at a discounted price. But caching has strict prefix-matching requirements—once the content or order of the tool list changes, the cache is immediately invalidated, and everything after the prefix must be recalculated at full price. This is exactly the tension the article points out: "reducing the number of tools" changes the tool list and thereby breaks the cache, while "keeping the tool list unchanged to hit the cache" contradicts on-demand streamlining. A viable compromise is often to fix a minimal tool set and then reuse the same one over a longer time window, so the cache discount truly materializes.

Background Supplement

Stateless API design is key to understanding this cost problem. Unlike the session mechanism of web applications, large model APIs don't store any context between requests on the server side. Each HTTP request is independent: the server receives your request, processes it, returns the result, and then completely forgets it. This means that "context" such as the system prompt, conversation history, and tool definitions is all maintained by the client, and reassembled and sent on every call. This design brings the convenience of horizontal scaling—any server can handle any request without session stickiness—but at the cost of repeated bandwidth and Token consumption. For a two- or three-turn conversation sent only once, this overhead is negligible; but for Agent loops that often run dozens of steps, the repeated transmission becomes a visible number on the bill.

Agent loops and the ReAct pattern are the direct reason this cost gets amplified. An Agent doesn't output an answer all at once; instead, it enters an iterative "reason—act—observe" cycle: the model first thinks about what to do next and outputs a tool call; the client executes that tool and appends the result to the history as a new message; then the entire history along with the tool list is sent to the model to let it decide the next step. This loop (commonly implemented as the ReAct pattern) is a complete API request at each step, so the number of tool Schema transmissions equals the number of steps. The cost of a single call isn't high, but once the number of loop iterations grows, the repeated portion is multiplied. Understanding this is what makes clear why "optimizing the tool list" has far greater leverage in Agent scenarios than in ordinary chat.

One-line Summary

API statelessness causes tool Schema to be retransmitted with every request, and 18 tools account for 36% of input Tokens in an Agent run.

Paragraph Summary

The article explains the cost problem of repeated AI Agent tool Schema transmission. Due to the stateless API design, the complete tool list must be sent with every request, and each step of the Agent loop is one request. Measured results show 18 tools consume 2,388 Tokens, accounting for 36% of input Tokens and 32% of total cost in a 10-step run, wasting 716 million Tokens per month at 1,000 runs per day. The author proposes three cost-reduction strategies: streamlining to task-relevant tools can significantly lower monthly costs, compressing description text can cut Tokens by another 35%, and using a tiktoken script to quantify your own overhead. The article also notes that counts are approximate, vendors rewrite JSON, and Prompt caching can reduce repeated costs but conflicts with switching tool sets; the core recommendation is to count your tools before optimizing.