Iris Search Agent: Backward Data Construction and SFT-RL Alternating Training Achieve Multi-Hop Reasoning Breakthrough

A search agent research project called Iris has recently attracted widespread attention in the AI community. The project released two models — Iris-mini and Iris-pro — trained at the 35B-A3B and 397B-A17B scales respectively, achieving state-of-the-art performance among open-source models across multiple search reasoning benchmarks.

It's worth noting that the parameter notation (e.g., 35B-A3B) reveals that these models are built on a Mixture-of-Experts (MoE) architecture. In the 35B-A3B case, 35B denotes the total parameter count of 35 billion, while A3B indicates that only 3 billion parameters are actually activated during each inference pass. The MoE architecture uses multiple parallel expert networks and a gating mechanism (Router), activating only a small subset of experts per forward pass. This allows the model to retain the capacity and knowledge of a large model while significantly reducing inference-time compute. This architecture has been widely adopted by models like DeepSeek and Mixtral, and represents a mainstream approach to balancing model capability with inference efficiency.

Backward Training Data Construction: From Hyperlink Structure to Multi-Hop Reasoning Chains

Iris's core innovation lies in its unique data construction pipeline. The research team starts from the hyperlink structure of web corpora and uses a backward construction approach to generate training tasks. Specifically, they extract entity graphs from seed pages and their outbound links, then construct multi-hop reasoning chains on top of these graphs.

To understand this, it helps to first grasp the concept of Multi-hop Reasoning: it refers to the model's ability to span multiple information sources and traverse a multi-step logical chain before arriving at a final answer. For example, answering "Which university did the founder of SpaceX graduate from?" requires the model to first identify that SpaceX was founded by Elon Musk, then retrieve his educational background. Traditional single-hop QA systems can only handle questions that are solvable with a single retrieval step, whereas multi-hop reasoning demands that the model plan search paths, integrate heterogeneous information, and maintain consistency across a reasoning chain — making it a central challenge in search agent research.

Hyperlinks in web pages inherently form a large-scale directed graph structure, where each page is a node and each hyperlink is an edge. This structure naturally encodes semantic relationships between entities — when one page links to another, it typically implies a topical or entity-level association between the two. The Iris team exploits this property by traversing hyperlinks from seed pages, extracting key entities, and building an entity relationship graph. The elegance of this approach is that it does not rely on pre-built knowledge graphs (such as Wikidata or Freebase), but instead mines structured relationships directly from the raw web ecosystem, ensuring that the data distribution closely mirrors real-world search scenarios.

To ensure genuine difficulty, the team applied descriptive rewriting to all non-answer entities, making them impossible to resolve through simple string matching. Descriptive rewriting replaces entity names with natural language descriptions of those entities. For example, replacing "Beijing" in a question with "the capital of China" prevents the model from taking shortcuts via keyword matching or entity name retrieval. This technique — known as shortcut removal in data augmentation and evaluation design — forces the model to perform genuine semantic understanding and reasoning.

Critically, only questions that the reference model cannot answer in a closed-book setting, but can correctly resolve when supporting evidence is provided, are retained. The closed-book filtering draws on the evaluation paradigm of knowledge-intensive NLP tasks: if a model can answer a question without external retrieval, the question is not challenging enough for that model; if the model still fails even after evidence is provided, the question exceeds its capability. This dual-filter ensures that training samples fall within the model's "zone of proximal development," maximizing learning efficiency.

SFT-RL Climbing: A Closed-Loop Training Strategy Alternating Supervised Fine-Tuning and Reinforcement Learning

On the training methodology side, Iris adopts an innovative strategy called "SFT-RL climbing," which alternates between Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL).

To appreciate the significance of this strategy, it helps to understand the respective roles of SFT and RL in large language model training. Supervised Fine-Tuning (SFT) involves further training a pre-trained model on labeled input-output pairs, teaching it the format and basic capabilities of a specific task. Reinforcement Learning (RL), on the other hand, uses reward signals to guide the model toward discovering better strategies through exploration — particularly well-suited for optimizing complex behaviors that are difficult to fully cover with static labeled data, such as multi-step search decisions and tool-calling sequences. In the LLM field, OpenAI's InstructGPT and the subsequent RLHF (Reinforcement Learning from Human Feedback) pipeline established the paradigm of following SFT with RL. Iris's innovation is in making these two phases alternating rather than sequential: high-quality behavioral trajectories discovered during the RL phase are fed back into the SFT phase as new supervised signals, forming a continuously improving positive feedback loop.

During the RL phase, the policy is optimized against live search, with reward judges and observation summarizers deployed directly within the training cluster to significantly improve training efficiency.

For excessively long rollouts, the system interrupts at the request level and resumes from the submitted prefix at the next step, avoiding wasted resources. In the context of reinforcement learning, a rollout refers to the complete interaction sequence executed by the model following its current policy — for a search agent, this encompasses the entire process from receiving a question to producing a final answer, including all search queries, web reading, and tool calls. In practice, rollouts for search agents can be very long, as the model may perform dozens of search queries and page reads. Iris's request-level interrupt-and-resume mechanism is analogous to checkpointing in operating systems: when a rollout exceeds its resource budget, the system saves the current state and halts; in the next training step, execution resumes from the saved prefix rather than discarding the already-completed computation. This design significantly reduces GPU resource waste and represents an important engineering practice for large-scale RL training.

After each RL training round, the research team feeds the most challenging yet most efficient rollouts back into the next round of supervised learning, forming a continuously improving closed loop.

Inference-Time Context Management: An Underappreciated but Critical Technique

The research team places particular emphasis on the importance of inference-time context management. They note that the performance gains from this technique exceed the differences between most systems reported in the literature, making it a broadly underappreciated capability.

The core challenge of context management is this: as a search agent executes multi-hop reasoning, it accumulates large volumes of search results and intermediate reasoning text, while the context window of a large language model is finite. How to retain the most critical information within the limited context space while discarding redundant content directly determines whether the model can access the necessary clues in subsequent reasoning steps. Effective context management requires a delicate balance between information retention and space conservation — especially critical for complex tasks that require ten or even dozens of search steps to complete.

Accordingly, each benchmark was evaluated under both context management enabled and disabled conditions, while keeping the toolset, context limits, and judgment criteria fixed. All results come from a single ReAct agent, without sub-agents or test-time verification, ensuring fairness and reproducibility of the evaluation.

ReAct (Reasoning + Acting) is an agent framework proposed by Yao et al. in 2023, whose core idea is to have the language model alternate between reasoning (Thought) and acting (Action). At each step, the model first outputs a reasoning passage describing its current thinking and next planned action, then executes a concrete action (such as a search query or web browsing), and then reasons again based on the returned observation. This framework naturally combines Chain-of-Thought with tool use and has become the mainstream architecture for search agents and tool-use agents. Iris's choice to use a single ReAct agent rather than a multi-agent collaborative system for evaluation demonstrates that its core capability improvements stem from the model itself rather than from complex systems engineering.

Benchmark Results: Comprehensive Leadership Among Open-Source Search Agents

With context management enabled, Iris-mini achieved scores of 82.2%, 84.8%, 86.9%, and 52.3% on BrowseComp, BrowseComp-ZH, DeepSearchQA, and HLE respectively. The larger Iris-pro further improved these to 88.6%, 85.1%, 92.9%, and 56.4%, representing the strongest overall performance among open-source search agents in their respective parameter ranges.

Benchmark	Iris-mini	Iris-pro
BrowseComp	82.2%	88.6%
BrowseComp-ZH	84.8%	85.1%
DeepSearchQA	86.9%	92.9%
HLE	52.3%	56.4%

Each of these benchmarks has a distinct focus. BrowseComp is a high-difficulty web browsing comprehension benchmark released by OpenAI, specifically designed to evaluate AI systems' ability to perform deep information retrieval and reasoning in real web environments; its questions are carefully crafted to be unsolvable through simple search or commonsense alone. BrowseComp-ZH is its Chinese counterpart, testing cross-lingual retrieval and reasoning capabilities. DeepSearchQA focuses on complex factual questions that require multi-step deep search to answer. HLE (Humanity's Last Exam) is an extremely challenging comprehensive knowledge test constructed by Scale AI in collaboration with domain experts — designed to sit at the boundary of human capability, with questions spanning expert-level problems across mathematics, science, humanities, and more. A score of 56.4% on this benchmark is already considered very high.

Notably, the BrowseComp-ZH results demonstrate that Iris also excels on Chinese-language search tasks, reflecting strong cross-lingual multi-hop reasoning capabilities. The score exceeding 90% on DeepSearchQA further validates the model's advantage in deep search reasoning tasks.

Open-Source Plans and Future Directions

The research team plans to release model weights along with a complete methodology covering data construction, training, and evaluation. This open-source commitment is highly significant for both academia and industry, and is expected to accelerate the iteration and deployment of search agent technology. Open-sourcing the complete methodology means that other teams can not only reproduce Iris's results, but also build on its data construction pipeline and SFT-RL climbing training paradigm to make further improvements and extensions — serving as an important catalyst for advancing the broader field of search agents.

The success of Iris demonstrates that through carefully designed backward data construction strategies and SFT-RL alternating training methods, there remains enormous room for improvement in search agents on multi-hop reasoning tasks. As the model weights and complete methodology are open-sourced, improvements and applications built on the Iris framework are well worth following closely.