Why Is Your Agent So Slow? Five Engineering Optimization Ideas

An Agent application used for performance reviews takes a long time for every query. The problem isn't the model itself—it's the tool-calling architecture. Based on a real troubleshooting process from a technical interview, this article breaks down the common root causes of Agent latency and the actionable optimization paths.

First, Locate: Which Architecture Does Your Agent Belong To

The first step in diagnosing performance issues isn't looking at code—it's confirming the architecture type. In the interview, the questioner initially described the system as "AI," and the interviewee Tanishq followed up with a key question: is this a RAG chatbot or an Agent application?

The answer was clear: this is a single-Agent system where users don't converse directly. Instead, it's driven by prompts and calls about 5 tools behind the scenes—GitHub, employee data (exposed via MCP), document queries, and more. The system uses API calls rather than a self-hosted LLM, and there's no separate memory layer—all context lives in the Agent's context.

This profile determines the optimization direction. Tanishq's summary: this system is essentially doing context engineering, with no additional memory architecture. That means every bit of wasted context window directly translates into latency and cost.

Is a Single Agent Enough

When the discussion later turned to multi-Agent approaches, Tanishq gave a clear verdict: for scenarios like performance reviews, a single Agent with a fixed set of tools is "more than enough." Introducing multiple Agents brings the complexity of memory management—both short-term and long-term memory need to be maintained—and the return on investment just isn't worth it. This point is worth noting: multi-Agent isn't a performance silver bullet; it's often a new source of latency.

MCP (Model Context Protocol) is a protocol proposed and open-sourced by Anthropic in late 2024 to standardize how LLM applications connect to external tools and data sources. It abstracts tool exposure into "Servers," and clients (Agents) discover and call tools through a unified interface, avoiding custom integrations for each data source. An MCP Server can host any number of tools, but the protocol itself doesn't specify whether calls are synchronous or asynchronous—that depends on the Server's specific implementation and underlying transport (such as stdio or HTTP/SSE). Therefore, when using MCP, you must verify whether the Server side truly supports concurrent processing; otherwise, even if the Agent framework supports async, actual calls will still degrade into serial waiting.

Context engineering refers to the practice of carefully selecting, organizing, and injecting the information a model needs within a limited context window. Unlike RAG, which relies on retrieval augmentation, a pure Agent architecture has no separate memory layer—all tool descriptions, historical results, and instructions are crammed into the same context window. The longer the context, the higher the inference latency and token cost. Therefore, reducing redundant tool descriptions and pre-storing static data are essentially all forms of context engineering.

Root Cause One: Synchronous Tool Calls Are the Biggest Bottleneck

The most direct finding from the interview was the issue of sync vs. async. The questioner admitted that the team hadn't considered async and was most likely using synchronous calls throughout.

Tanishq explained the consequences: suppose there are 5 tools and a certain use case requires calling all of them. In synchronous mode, T2 must wait for T1 to return before it can start, then T3 waits for T2… and the waiting time accumulates linearly. For scenarios like performance reviews that need to pull from multiple sources simultaneously—employee data, code commit records, documents—this is the most typical latency amplifier.

The optimization direction is clear: if tools have no output dependencies between them, they should run asynchronously in parallel; only when a later tool depends on the result of an earlier one does execution need to be sequential. This is a P1-level issue—fix it first.

The difference between synchronous and asynchronous tool calls lies in whether the control flow blocks. With synchronous calls, the Agent must wait for a response after initiating a tool request before it can continue, and multiple independent tools can only execute serially, with total latency equal to the sum of each tool's latency. Asynchronous calls allow multiple requests to be initiated simultaneously, scheduled by an event loop or concurrency framework, with total latency roughly equal to the latency of the slowest tool. The prerequisite for implementing async is that there are no data dependencies between tools, and both the Agent framework and the tool server support concurrency. In mainstream frameworks like LangChain and LlamaIndex, asynchronous tools require explicitly using async functions or the ainvoke interface; otherwise, the default path is synchronous.

Root Cause Two: The MCP Server Exposes Too Many Tools

This system uses an MCP Server to expose tool capabilities. Tanishq pointed out an easily overlooked detail: the MCP Server itself doesn't guarantee async—you need to confirm whether the specific implementation supports async.

The more insidious problem is tool count bloat. A company-level MCP Server might host 10 tools, but a given department actually only needs 5 of them. If you call without filtering, you pay a double cost: first, the extra tool calls add latency; second, the tool descriptions consume more context, and since the system has no separate memory layer, context bloat directly slows down inference. Tanishq's phrasing was "one solution to two problems"—filter tools before calling the MCP Server.

Root Cause Three: Data That Should Be Cached Is Being Fetched in Real Time

Not all tools are worth calling. Take employee data as an example—Tanishq pointed out that this kind of data doesn't change every day; only some fields get updated. Information with a fixed pattern, like remaining vacation days, could be stored in the backend before calling tools and injected directly into the system prompt or passed to the Agent as additional instructions.

This leads to a more general criterion: some tools do very "mundane" things, and their output could be placed directly into the model's context in advance—it's not worth going through the full tool call-response chain. Eliminating one round trip eliminates one segment of latency.

Can the Tools Themselves Be Cut

A related question that needs answering: do you really need 5 tools? Tanishq suggests reviewing the tool set at the architecture design stage and removing from the tool list any capabilities that context can provide directly. Fewer tools means a smaller decision space for the Agent and shorter call paths.

The round-trip overhead of tool calls includes not only network transmission but also the complete loop of the model generating tool call parameters, parsing responses, re-injecting results into context, and inferring again. Every tool call implies at least one additional LLM inference step, and LLM inference itself is the biggest latency contributor. Therefore, pre-storing low-change-frequency data in the backend and injecting it directly into the system prompt is equivalent to replacing the full chain of "generate call → network request → parse → re-infer" with a single context write, and the latency saved is often several times the pure network time. The criterion can be simplified to: if a tool's output remains essentially unchanged for the same user over a period of time and its volume is manageable, pre-injection should be preferred over real-time calls.

Optimization Ideas: Tool Composition and Batching

During the interview, the questioner proposed two ideas from software engineering intuition, and Tanishq gave his assessment of each.

Tool composition: if two tools' tasks have significant overlap—for example, both pull employee-related data from the same server—they can be merged into one tool that does both things at once. The applicable condition is high overlap; if the tools are completely independent, the correct approach is async rather than merging.

Batching: this idea is common in traditional backends—combining multiple requests into a batch. But Tanishq pointed out that for Agents, "multiple Agents calling the same tool" implies a multi-Agent architecture, which brings complexity like memory maintenance. For the current single-Agent performance review scenario, introducing multi-Agent for this purpose is not recommended.

An Actionable Optimization Checklist

Organizing the judgments from the interview into action items:

• Check whether all tool calls are synchronous, and convert independent tools to async parallel (highest priority)
• Confirm whether the MCP Server implementation supports async rather than assuming by default
• Filter the MCP Server's tool list before calling, keeping only the tools needed for the current scenario
• Pre-store low-change-frequency data (like vacation balance) in the backend and inject it directly into context
• Review the tool set and cut tools whose output can be provided directly by context
• Consider merging two tools with high task overlap
• Avoid introducing multi-Agent architecture unless truly necessary

The core logic behind this approach is consistent: Agents are slow not because the model is slow, but because the architecture contains unnecessary serial waiting, redundant calls, and context waste. Measure first, then rearrange the call graph according to dependencies—latency usually improves significantly.

The interview ended with a light-hearted exchange—the questioner asked whether there's a tool for "a raise after the performance review," and the answer was "there's a tool for a pay cut, though." Beyond the joke, the real takeaway is that follow-up question: first figure out whether you're dealing with RAG or an Agent, then talk about optimization.