OpenAI Agents SDK Hides 200 Tools, Cuts Token Consumption by 9x

The Problem: Too Many Tools, Out-of-Control Token Costs

When building AI agents, one often-overlooked cost trap is the Token cost of tool definitions themselves. When your agent has 200 tools attached, every request stuffs all the tools' descriptions (JSON Schemas) into the model's input—even if the task only needs one of them.

According to a demonstration by a YouTube tech channel (Yash's OpenAI Agents series), the core of this problem is: before the model even sees the user request, it's already forced to "read" the full descriptions of hundreds of tools. For large systems with multiple agents and multiple tools, this overhead quickly accumulates into a substantial bill.

JSON Schema and Token Consumption: Under the OpenAI Function Calling spec, each tool is described by a structured JSON Schema containing metadata like the tool name, functional description, parameter names, parameter types, whether each parameter is required, and enum value ranges. A medium-complexity tool (say, with 5–8 parameters and nested objects) typically has a Schema of 150–400 Tokens. The full definitions of 200 such tools average about 30,000–80,000 Tokens, and this content is fully re-transmitted in every conversation request—because large models are stateless and don't retain the previous turn's context, so all history and tool definitions must be reassembled and sent by the client. This means tool definition overhead isn't a one-time cost, but one that accumulates linearly with the number of requests, and is especially pronounced in multi-turn agent loops.

The Solution: Tool Search + Namespace Grouping

The OpenAI Agents SDK provides two key mechanisms to address this: the Tool Search Tool and Tool Namespace.

The core idea is straightforward—hide all tools by default, and only when the model truly needs one does a dedicated search tool "find" the right one. According to the video's analysis, even with 200 tools hidden, the model only needs one extra search call to locate the target, rather than paying the Token cost of all tools every time.

Implementation Mechanism: The defer_loading Parameter

The implementation relies on the key parameter defer_loading. As the channel explained in its code demo, when defer_loading=True, the tool is completely invisible to the model; only when the Tool Search Tool finds it based on the task description does its information enter the model context.

The approach in the demo is: build a tool factory function in which the tool name is customized via name_override (e.g., check_mission_005), and pass in the defer_loading parameter to control visibility. All tools that need to be hidden are set with defer_loading=True, and finally a separate tool_search_tool is added to handle retrieval.

Background on Function Calling and Tool Routing: The traditional approach hands all available tools to the model at once, letting the model choose which one to call based on user intent. The advantage of this "full exposure" strategy is simple logic with no extra retrieval step; the downside is that the model must "read through" all tools every time. The Tool Search mechanism essentially splits this into two levels: the first level has the search tool retrieve candidates from the tool library based on task semantics, and the second level injects the small number of matched tools into the context for the model to formally call. From an information retrieval perspective, this is like replacing a "brute-force full scan" with "coarse recall first, then fine ranking," trading one extra routing call for a massive compression of context. The trade-off to weigh is that if retrieval recall is inaccurate, the model may not get the correct tool, so the semantic clarity of tool names and namespace descriptions directly affects the final result.

Namespace Grouping: Reducing the Search Scope

Even if 200 tools are hidden, retrieval efficiency will drop if the search space is too large. The video introduces namespace grouping as a further optimization.

Specifically, the 200 tools are divided into 10 namespaces of 20 tools each, with each namespace having its own name and description. When the model needs to find a tool, it first locates the relevant category among the 10 namespaces, then does an exact match among the 20 tools—shrinking the search scope from 200 down to 20.

According to the video demo, the implementation uses a loop to generate 10 ToolNamespace instances, each containing 20 build_mission_tool instances, which are finally returned to the Agent together with the tool_search_tool.

The Relationship Between Token Budget and Context Window: A large model's context window is limited (e.g., 128K Tokens), and tool definitions, system prompts, conversation history, retrieved documents, and user input all share this space. Bloated tool definitions not only drive up the bill but also crowd out window capacity that could be used for business content, and can even trigger truncation that loses critical information. Beyond speeding up retrieval, namespace grouping has an implicit value: it makes the tool library's organizational structure "visible" to the model, so the model can first understand the system's capability boundaries from namespace descriptions, then decide whether to go deeper. For enterprise-grade agents with hundreds of tools, this hierarchical metadata description is itself a low-cost "capability index."

Real-World Test: Tokens Drop from Tens of Thousands to 943

The comparison experiment in the video is compelling. Two configurations use the same query, "what is the status of mission 005":

• Plain Agent: all 200 tools visible, no grouping, no Tool Search. Since all tool definitions are carried in the context, input Token consumption is extremely high.
• Tool Search Agent: all 200 tools hidden, exposing only the Tool Search Tool, combined with namespace grouping.

The test results show that after enabling Tool Search, the input Tokens for the first LLM call are only 943. According to the video author's estimate, this is about 5–9x cheaper than the unoptimized version.

The combined Tokens of the two LLM calls (one to decide which tool to call, one to generate the final answer) achieve substantial savings compared to the Plain Agent approach. In the video demo, the Agent correctly returned the result "mission 005, danger level 3, standard clear procedure," showing that functionality was not compromised by hiding the tools.

Benefits and Use Cases

The benefits of this approach go beyond saving money. According to the video's analysis, reducing Tokens brings three chain effects:

1. Lower cost: input Tokens drop directly, and the bill follows.
2. Improved latency: shorter input means faster inference and reduced response time.
3. Scalability: as both the number of agents (N) and tools (N) grow, Tool Search keeps the system within a controllable cost range.

Applicable scenarios include: enterprise-grade agents with a large number of tools, multi-agent collaboration systems, and any project with more than 20 tools that needs to control its Token budget. Note that Tool Search is not zero-cost—it introduces one extra search call—but when there are enough tools, this overhead is far smaller than transmitting all tool definitions every time.

Conclusion

The OpenAI Agents SDK's Tool Search feature hides tool definitions via defer_loading and narrows the search scope through ToolNamespace grouping, compressing input Tokens by nearly an order of magnitude while preserving full tool-calling capability. For agent systems with ballooning tool counts, this is a practical optimization worth incorporating into architectural design.