Agentic Search in Practice: A Guide to Selecting and Combining Search Tools for Agent Context

In the era of AI Agents, Context Engineering has become the core factor determining Agent output quality. Context Engineering refers to the engineering practice of systematically designing, managing, and optimizing the contextual information fed to large language models — unlike Prompt Engineering, which focuses on wording optimization for single interactions, Context Engineering concerns itself with how to dynamically collect, filter, compress, and organize information from multiple sources throughout an Agent's entire runtime, fitting it within the model's context window limits while maximizing information relevance and effectiveness. As mainstream model context windows have expanded from 4K to 100K tokens and beyond, the focus of Context Engineering has shifted from "how to cram in more information" to "how to ensure the right information appears in the right place."

Elastic engineer Leonie shared a comprehensive practical methodology for Agentic Search at the AI Engineer conference — starting from the limitations of traditional RAG, she progressively demonstrated how to build more powerful search tool combinations that help Agents precisely retrieve information from multiple context sources.

She proposed a bold viewpoint: 80% of Context Engineering work is Agentic Search. This isn't just about retrieval — it's about how to let Agents autonomously decide what to search for, which tools to use for searching, and how to process search results.

From RAG to Agentic RAG: The Evolution of Search Paradigms

Inherent Flaws of Traditional RAG

Retrieval-Augmented Generation (RAG) was proposed by Meta AI in 2020. Its core idea is to combine parametric knowledge (knowledge stored in model weights) with non-parametric knowledge (external document repositories). The traditional RAG workflow includes: an offline phase where documents are chunked and converted into vectors via embedding models for storage in vector databases; and an online phase where user queries are similarly vectorized, similar document chunks are retrieved through Approximate Nearest Neighbor (ANN) algorithms, concatenated, and fed into the LLM for answer generation.

The original RAG approach used a fixed retrieval pipeline: the user message was passed almost verbatim as a vector search query, document chunks were pulled from the database, and then sent to the LLM alongside the user message. While this effectively mitigated LLM knowledge cutoff and hallucination issues, its fixed pipeline architecture is inherently unsuitable for complex task scenarios requiring multi-step reasoning and dynamic decision-making. This manifests as two clear problems:

1. The system executes retrieval regardless of whether additional context is needed, which in the worst case actually interferes with the LLM's judgment
2. A single retrieval cannot handle multi-hop reasoning — when the first round of retrieval results reveal new search leads, the system cannot perform a second round of searching

What is multi-hop reasoning? Multi-hop reasoning refers to a reasoning process that requires traversing multiple intermediate steps or multiple knowledge sources to arrive at a final answer. For example, answering "Who founded the university that Tesla's CEO graduated from?" requires first retrieving "Tesla's CEO is Elon Musk," then retrieving "Musk graduated from the University of Pennsylvania," and finally retrieving "The University of Pennsylvania was founded by Benjamin Franklin." Traditional single-pass RAG retrieval cannot handle this type of question because the first round of retrieval results often reveal new search leads that require dynamically adjusting subsequent query strategies.

The Breakthrough of Agentic RAG

Agentic RAG replaces the fixed pipeline with search tools, letting the Agent autonomously decide whether to invoke search, when to rewrite queries, and whether multiple retrieval rounds are needed. However, this still involves only a single context source.

In real-world Context Engineering scenarios, context is scattered across multiple locations: local file systems (code files, planning documents, Agent skill files), databases, the web, long-term memory, and more. Each context source requires a corresponding search tool — file search tools, skill loading tools, semantic search tools, general query tools, web search tools, memory tools, and even Shell tools.

Three Fundamental Principles for Building Good Search Tools

Agentic Search looks simple on the surface: the user makes a request, the Agent calls the right tool with the right parameters, and returns results. But in reality, it can fail in multiple ways.

Failure Mode One: Agent Doesn't Call Any Tool

The Agent may believe it can answer the question using only its parametric knowledge, completely skipping context retrieval. This is especially dangerous in scenarios requiring real-time data or private knowledge.

Failure Mode Two: Agent Calls the Wrong Tool

Leonie mentioned a colleague's experience — the biggest challenge in their project turned out to be getting the Agent to call the database search tool instead of the web search tool. When the number of tools increases, this "wrong tool selection" problem becomes increasingly severe.