How Do AI Agents Orchestrate Multiple Tools? MCP and Dynamic Discovery Explained

Title

How Do AI Agents Orchestrate Multiple Tools? MCP and Dynamic Discovery Explained

Meta Description

Former AWS architect Raj explains how AI agents orchestrate multiple tools: MCP protocol dynamic discovery, ReAct loop decision-making, and vector retrieval to optimize context bloat. Learn the core mechanisms and best practices of tool orchestration.

Article Content

When an AI agent is connected to multiple tools, how does it know which one to call and in what order? This is a common interview question and a key challenge in real-world development. Former AWS Principal Solutions Architect Raj breaks down the underlying mechanisms in a video and explains how the MCP protocol moves tool orchestration from hardcoding to dynamic discovery.

The Basic Architecture of Tool Orchestration: Code as the Hub

The user's input prompt is not sent directly to the large language model (LLM), because the LLM itself cannot connect to external tools. The prompt is first sent to a piece of code, which is responsible for interacting with the LLM and managing all tool connections. Tools are connected through MCP (Model Context Protocol) servers, and an MCP client runs within the code.

The code first sends a tools/list request to each MCP server to obtain the tool's name, description, and input parameter schema. For example, a weather tool would return: name get_weather, description "Get current weather information", with input requiring a location field (string type, required). This process is completed dynamically at runtime and does not need to be hardcoded in advance.

Subsequently, the code packages the user prompt together with all tool information and sends it to the LLM. The LLM determines which tools need to be called based on semantic understanding, rather than relying on static rules in the code (such as "for travel, use tools 1-4"). Raj emphasizes: "The LLM is smart enough that it is basically doing semantic search."

MCP (Model Context Protocol) is an open protocol proposed by Anthropic at the end of 2024, aiming to unify the way LLMs connect to external data sources and tools. It adopts a client-server architecture: MCP servers encapsulate specific capabilities (such as accessing databases, calling APIs, reading files) and expose tools, resources, and prompt templates through standard interfaces; MCP clients run in the agent-side code and are responsible for establishing sessions with one or more servers. The protocol transmits messages based on JSON-RPC 2.0. Core methods include tools/list (enumerating available tools and their JSON Schema parameter definitions), tools/call (executing a specified tool and returning the result), and the list_changed event used to notify capability changes. It is precisely this standardized interface that enables tools to be discovered and invoked at runtime without having to hardcode each tool's signature and adaptation logic in the code, and it also allows tools from different vendors to be integrated into any MCP-compatible agent in a unified way.

The ReAct Loop: Alternating Reasoning and Action

After the LLM decides to call a tool, the code executes the actual call and returns the result to the LLM. The LLM continues reasoning based on the new information, which may trigger the next tool call. For example, when planning a trip to New York, the LLM first calls the weather tool, finds that the weather is good, then calls an activity recommendation tool to get a list of outdoor activities, and then calls a hotel price tool to compare options. This "reasoning-action" loop (ReAct) continues until the LLM believes the information is sufficient, at which point it returns the final answer to the user.

The advantage of this approach is flexibility: adding new tools requires no changes to the code logic. But the drawback is also obvious—context bloat. On each request, the code stuffs the descriptions and parameters of all tools into the LLM's context, even if only a few tools are relevant. Raj points out that a 100-token prompt may thus expand to 1000 tokens, leading to higher costs.

ReAct (Reasoning + Acting) is a prompting and execution paradigm that lets language models alternate between reasoning and external actions, proposed by Yao et al. in 2022. Its core is to organize the model's output into a "thought-action-observation" loop: the model first reasons in natural language about the current state and the next goal, then selects an action (such as calling a tool and providing parameters), the execution environment returns an observation result, and the model continues the next round of reasoning based on it. Compared with generating an answer in one shot, ReAct allows the model to proactively obtain external information when information is insufficient and to revise its plan across multiple steps. In agent implementations, this loop is usually driven by code: the code repeatedly sends the conversation history and tool execution results back to the LLM until the model outputs a termination signal or reaches the maximum number of steps. The cost of ReAct is that the accumulated context must be resent in every round; the more rounds there are and the longer the tool descriptions are, the higher the token consumption and latency become. This is also an important motivation for the context optimization discussed later.

Advanced Approach: Tool Search Based on Vector Retrieval

To solve context bloat, a more advanced approach borrows the idea of RAG (Retrieval-Augmented Generation). The metadata of all tools (name, description, input schema) is stored in a vector database or in memory. When a user prompt arrives, the code converts it into a vector, performs semantic search in the tool library, and only recalls the information of relevant tools to send to the LLM.

For example, for the prompt "plan a trip to New York based on the weather," semantic search may recall only four tools—weather, activities, hotels, and taxes—while ignoring irrelevant AWS log tools. In this way, the LLM's context contains only the necessary tools, greatly saving tokens. Raj mentions that Claude may use a similar mechanism called "tool search," but the storage details have not been disclosed.

This approach has not yet been widely adopted because it requires additional implementation on the agent side. However, as the number of tools grows (to hundreds), it becomes very effective. The MCP protocol itself also standardizes tool invocation: execution is performed by specifying the tool name and parameters through tools/call, which simplifies integration.

RAG (Retrieval-Augmented Generation) refers to the practice of retrieving relevant content from an external knowledge base before generation and splicing it into the prompt. Its typical process is: use an embedding model to split documents into chunks, encode them into vectors, and store them in a vector database; during querying, encode the user's question into a vector in the same space, and recall the most relevant fragments through cosine similarity or an approximate nearest neighbor (ANN) index, then provide them to the LLM as context. Transferring this idea to tool selection means encoding each tool's name, description, and parameter schema into vectors to build a "tool index," and using the user prompt to retrieve the most matching subset of tools. Unlike document retrieval, the goal of tool retrieval is to recall "tools that can complete the task" rather than "text that contains the answer," so the quality of tool descriptions directly affects recall performance—tools with vague descriptions are more likely to be missed.

Practical Insights and Optimization Recommendations

For developers, Raj's advice is: if certain tools are not needed, disconnect them to avoid context pollution. At the same time, you can enable tracing in the agent to observe the back-and-forth interaction between the code and the LLM and understand the orchestration order. In the future, as the tool ecosystem expands, dynamically retrieved tool selection may become standard.

The essence of tool orchestration is to let the LLM make decisions at runtime rather than writing fixed rules. MCP provides standardized discovery and invocation interfaces, while semantic retrieval optimizes efficiency. The combination of the two is driving AI agents from simple tasks toward complex workflows.

One-line Summary

MCP lets agents dynamically discover tools, LLMs semantically orchestrate them via the ReAct loop, and vector retrieval can mitigate context bloat.

Paragraph Summary

The article breaks down the orchestration mechanisms of AI agents in multi-tool scenarios. The user prompt first enters the code hub; the code sends tools/list to each MCP server through the MCP client to obtain the tool's name, description, and parameter schema, then packages them with the prompt for the LLM, which decides which tools to call and in what order based on semantic understanding rather than static rules. The call results are returned to form the ReAct "reasoning-action" loop until the information is sufficient. This dynamic approach is flexible, but the cost is context bloat: all tool descriptions enter the context, and a 100-token prompt may expand to 1000 tokens. An advanced approach borrows from RAG, storing tool metadata in a vector database and recalling only relevant tools based on prompt semantics, greatly saving tokens. Practical advice is to disconnect unneeded tools and enable tracing to observe the orchestration process.