How Uber Lets AI Call 5,000 Tools Without Blowing Up the Context Window

When an AI agent is faced with more than 5,000 tools, what happens? The standard approach is to list the full definition of every tool for the model — which stuffs the context window so full that the budget is exhausted before the agent even starts working. Uber's answer is a proxy solution called Omni MCP, built around one core idea: don't make the agent load all tools at once. Instead, fetch them on demand, step by step.

The Root of the Problem: Too Many Tools, Too Little Context

The standard MCP (Model Context Protocol) client behavior is to list the definition of every tool — name, description, and input parameters. With just a handful of tools, this is fine. But at Uber's scale, definitions for 5,000+ tools will directly fill up a model's context window.

When the context is consumed by tool definitions, two things happen: the space available for actual task reasoning shrinks, and the token cost per call spikes dramatically. For production-grade agentic systems that need to call tools at high frequency, this is an unavoidable engineering bottleneck.

MCP (Model Context Protocol) is an open protocol introduced by Anthropic in late 2024, designed to provide a standardized interface between AI models and external tools and data sources. It's often described as the "USB-C of AI" — unifying the way agents call tools so that any MCP-compatible model can use tools from any vendor directly. A standard MCP client calls the tools/list endpoint on initialization, fetching the complete definitions (in JSON Schema form) of all available tools on the server in one shot. These definitions are then injected into the model's context via system prompt or tool declarations. For models like GPT-4o or Claude, context windows typically run between 128K and 200K tokens, while a single tool definition can occupy anywhere from dozens to hundreds of tokens. Stack 5,000 of them together and you easily exceed one million tokens — far beyond the limit of any mainstream model.

Omni MCP: Four Small Tools Instead of 5,000 Definitions

Uber's solution is to introduce a proxy called Omni MCP. Rather than exposing all 5,000 tools directly to the agent, it gives the agent just four small tools to use in sequence: discover a server, list that server's tools, fetch the schema (input definition) for a specific tool, and finally execute the tool.

The entire flow looks more like a progressive retrieval process:

• The agent needs to query "order status"
• It first locates the relevant orders server
• Asks that server to list its tools
• Fetches the schema for just one tool
• Runs only that one tool

The key point: only these results enter the context. The agent doesn't need to know upfront what all 5,000 tools look like — it drills down layer by layer only when it actually needs to. This transforms "one-shot full load" into "on-demand incremental load."

This "on-demand incremental loading" approach maps directly to a classic software engineering pattern — Lazy Loading. In traditional database ORM frameworks or front-end asset loading, lazy loading defers the initialization of objects or resources until they are actually needed. Omni MCP transplants this pattern into the tool orchestration layer: the agent starts with only a "directory of directories," and each query fetches only the minimum information needed for the current decision. This shares the same core logic as Retrieval-Augmented Generation (RAG) — RAG doesn't stuff the entire knowledge base into the context; it uses semantic search to retrieve relevant fragments on demand. Omni MCP's four-step method can be thought of as a tool-level RAG: discovering the server is coarse ranking, listing tools is fine ranking, fetching the schema is reading a document fragment, and execution is the final "citation."

Gateway-Layer Permissions and Response Projection

Omni MCP does more than just save context — it handles two additional concerns at the gateway layer.

Permission checks: When a tool is actually called, the gateway verifies whether the caller has the required permissions. However, the presentation specifically notes that Uber did not clarify whether tools the caller lacks permission to use are hidden from the tool listing itself — meaning it's unclear whether permission filtering happens at the "list" stage or only at the "execute" stage.

Response Projection: The agent requests only the fields it needs, and the gateway strips out everything else from the response. This further reduces the amount of irrelevant data entering the context, and is Uber's own term for this mechanism.

The term "Response Projection" borrows from the concept of "projection" in database queries — in SQL, writing SELECT field1, field2 instead of SELECT * is a projection operation that returns only the columns the requester actually needs. Uber applies this idea to tool responses: the agent declares which fields it cares about when calling a tool, and the gateway actively strips out irrelevant fields from the downstream service's full response before returning it to the model. This is especially impactful for APIs with large response bodies — an order query endpoint might return a JSON object with dozens of fields, but the agent's current task may only need status and estimated_time. After projection, the data entering the context might shrink from thousands of tokens to just tens of tokens, and those savings accumulate significantly in high-frequency call scenarios.

No Quantified Numbers Published — Should You Copy This?

One thing worth noting with a clear head: Uber has not published any measured savings data. This architecture is logically sound, but how many tokens are actually saved and how much latency is added — those numbers are absent from the official account.

Whether to adopt this approach, therefore, depends on your own scale:

• If your tool list is large, it's worth testing what the four-step flow (discover → list → fetch schema → execute) actually delivers in practice.
• If you only have a small single server, stick with the standard tool listing — adding a proxy layer just introduces unnecessary extra round trips and complexity.

Implications for Agent Engineering

Omni MCP represents an engineering philosophy for handling tool scalability within the MCP ecosystem: treat tool discovery as a retrieval problem, not a loading problem. When the tool library grows to hundreds or thousands of entries, the "inject everything into context at once" pattern is bound to break down. Layered lazy loading combined with gateway-side projection will become the more sustainable architectural choice.

This approach also reminds us that performance optimization in agentic systems doesn't live in the model alone — much of it happens in the "plumbing" layers of tool orchestration, context management, and gateway design. The real engineering value is often hiding in these unglamorous middleware layers.