Retriever-Free Retrieval: DecisionRAG Navigates Documents with Decision Models

Retrieval-augmented generation (RAG) systems often bottleneck not at the generation side, but at the retrieval side. Similarity matching or lexical matching in a fixed embedding space determines which documents can be fed into the language model. Once the retriever misses key passages, even the strongest generation model struggles to compensate. This "capability asymmetry" has spawned a new idea: can the language model itself become the retriever?

The Bottleneck of Traditional Retrieval: Asymmetry in Fixed Embedding Spaces

Current mainstream retrieval approaches rely on two paths: lexical matching (e.g., BM25) or dense vector similarity. Both operate in a predefined representation space, and the ceiling of retrieval quality is determined by that space. The problem is that this space is not aligned with the generative model's understanding capability—the generative model can discern subtle semantic relationships between passages, but the retriever only performs a one-shot vector comparison.

According to the original analysis, the essence of this asymmetry is: the retriever is "blind-selecting," while the generator is "close-reading." The retriever cannot leverage the language model's deep understanding of context, intent, and document structure, relying only on surface-level similarity. This explains why in complex question-answering scenarios, retrieval recall often becomes the ceiling of the entire system.

BM25 is a classic retrieval algorithm based on term frequency. It scores documents by counting the frequency of query terms in documents (TF) and inverse document frequency (IDF), combined with document length normalization. Its core assumption is: documents with more occurrences of query terms are more relevant. BM25 does not rely on semantic understanding, so it is insensitive to lexical variants, synonyms, and context, but it is computationally efficient and highly interpretable, long serving as a retrieval baseline. Dense vector similarity maps queries and documents into a low-dimensional dense space via neural networks, measuring relevance with cosine similarity or dot product; its quality heavily depends on the embedding model's training data and objectives, and retrieval and generation models are typically trained independently, leading to misaligned representation spaces. Both belong to the "single comparison" paradigm and cannot perform multi-step reasoning like generative models.

DecisionRAG's Core Mechanism: Joint Evaluation of Candidate Branches in a Single Forward Pass

DecisionRAG proposes a "retriever-free" retrieval paradigm. It does not rely on a first-stage retriever, nor does it generate any tokens; instead, it uses a decision model to jointly evaluate multiple candidate branches in a single forward pass. The key insight is: when multiple candidates are judged together in a shared context, their relevance estimates become comparable.

This is similar to how a person finds a book in a library: first look at the broad category, then narrow down to the shelf, and finally flip to the specific passage. DecisionRAG navigates along the inherent structure of the corpus—descending layer by layer from broad topics to individual passages, deciding at each level which branches are worth exploring further. This hierarchical navigation allows the model to leverage the semantic understanding already present in pretrained language models without requiring task-specific training.

Why Shared Context Improves Relevance Judgment

When evaluating each candidate individually, the model lacks a reference frame, and relevance scores tend to drift. Placing candidate branches in the same context for joint judgment allows the model to calibrate the scale through comparison, making scores across different branches relatively comparable. This is the key design that distinguishes DecisionRAG from traditional pointwise reranking.

Pointwise reranking is a common paradigm in information retrieval: computing a relevance score for each candidate document independently, then sorting by score. Its drawback is the lack of relative comparison among candidates; the model struggles to calibrate the scale of absolute scores and is susceptible to document length, domain shift, and other factors. In contrast, pairwise or listwise reranking feeds multiple candidates into the model simultaneously, letting the model judge relative relevance through comparison. DecisionRAG's shared-context joint evaluation is essentially a listwise judgment, leveraging the language model's self-attention mechanism so that candidate branches reference each other within the same context window, thereby obtaining more stable and comparable relevance estimates.

Empirical Performance: Competitive Without Training or a First-Stage Retriever

According to the original data, without task-specific training and without a first-stage retriever, DecisionRAG outperforms dense retrieval with an 8B reranker by 13.8 R@1 points on MultiDoc2Dial and by 17.4 Hit@1 points on Natural Questions. Tests cover four benchmarks, including a corpus of 2.68 million Wikipedia passages.

Efficiency is also noteworthy: retrieval takes only 0.7 to 1.8 seconds per query. Considering it skips the first-stage recall of the traditional two-stage pipeline, this latency level is valuable for practical deployment.

R@1 (Recall at 1) and Hit@1 are commonly used metrics in retrieval evaluation: the former is the proportion of queries where the top-ranked result contains the correct answer, and the latter is the proportion where the top-ranked result hits a relevant document. Both focus on accuracy at the top-1 position; higher values indicate the retriever's stronger ability to rank the most relevant result first. In multi-document question-answering scenarios, R@1 and Hit@1 directly reflect whether the system can feed key passages to the generator, and are therefore often used as proxy metrics for the recall capability of end-to-end RAG systems. DecisionRAG's improvements on these metrics mean it reduces reliance on separate embedding indexes and rerankers.

Implications for RAG Architecture

DecisionRAG's significance goes beyond a new retrieval method; it points to an architectural reflection: must retrieval and generation be separated? When language models are powerful enough, delegating retrieval decisions to the model itself may be more flexible than maintaining a separate embedding index.

Of course, this path still has open questions—scalability on large-scale corpora, compatibility costs with existing indexing systems, and the robustness of decision models under domain shift all require more independent reproductions. But the direction of "giving the retriever the language model's understanding" has already provided a convincing preliminary answer.