RAG Hybrid Retrieval: Does Combining BM25 and Vector Search Improve Evidence Retrieval?

Your RAG application retrieved five chunks that all look relevant, but none of them actually answers the question. The LLM still confidently produces an answer—a fluent response built on the wrong evidence. Before tweaking prompts or switching models, ask yourself: did the retriever find the evidence needed to answer the question?

That is the core focus of this article. We will compare BM25, vector search, and hybrid retrieval using Reciprocal Rank Fusion (RRF) to see their respective strengths and limitations.

The Strengths and Blind Spots of Vector Search

Vector search looks for semantically similar passages through embedding vectors, matching even when the wording differs. For example, between the question "How do I reset my password?" and the document "Recover account access by updating your credentials," semantic similarity helps establish the connection.

But consider this query: "How do I fix error AUTH_403?" Vector search might rank generic login troubleshooting advice ahead of the passage containing the specific solution. Although it can also retrieve exact identifiers, semantic similarity does not guarantee that the most useful evidence ranks first.

Vector search excels at semantic matching, but in scenarios requiring exact keywords, the ranking may not be ideal.

Embedding vectors are dense vector representations that map text (words, sentences, or passages) into a high-dimensional space, typically generated by pre-trained language models (such as BERT, Sentence-BERT, or dedicated embedding models). The core idea is that texts with similar semantics are also close in vector space, so semantic relevance can be measured via cosine similarity or dot product. Embedding vectors can capture synonym substitutions, paraphrases, and contextual information, which makes them perform excellently in semantic retrieval. However, embedding models are sensitive to the distribution of their training data, and their representations of rare words, proper nouns, and exact identifiers (such as error codes) may not be sufficiently stable, causing this key information to be diluted or ignored in similarity computation. In addition, embedding vectors are usually high-dimensional (hundreds to thousands of dimensions), so computational and storage costs must also be taken into account.

Where BM25 Excels

BM25 ranks documents based on keyword matching, accounting for term frequency, term rarity, and document length. It performs well in the following scenarios:

• Exact error codes, such as AUTH_403
• Product names and technical terms
• Queries where specific words are critical

BM25's weakness is vocabulary mismatch. When the keywords in the question do not overlap with the terms in the document—for example, "reset password" versus "credential recovery"—it may miss relevant passages.

Method	Primary Strength
BM25	Keyword matching
Vector search	Semantic similarity

The two are complementary, but neither alone can guarantee finding the right evidence.

BM25 (Best Matching 25) is a ranking function based on the probabilistic information retrieval framework. Its core idea is that the relevance contribution of a query term to a document depends on the term's frequency in the document (TF), the term's rarity across the entire corpus (IDF), and a document length normalization factor. Specifically, the TF component uses a saturation function to avoid over-weighting from unbounded term frequency growth; the IDF component assigns higher weight to rare terms because rare terms better discriminate between documents; and document length normalization prevents long documents from gaining an unfair advantage simply by containing more words. The BM25 formula has two tunable parameters, k1 and b, which control the strength of term frequency saturation and length normalization, respectively, typically set to k1≈1.2–2.0 and b≈0.75. BM25 does not rely on semantic understanding and is based entirely on term statistics, making it extremely effective for exact keyword matching, but it cannot handle vocabulary mismatch (i.e., cases where query terms and document terms do not overlap but are semantically related).

How Hybrid Search Uses RRF

Hybrid search combines keyword retrieval and vector retrieval. One common way to merge the results is Reciprocal Rank Fusion (RRF):

RRF(d) = Σ 1 / (k + rankᵢ(d))

Where:

• d is a document or chunk.
• rankᵢ(d) is its position in each result list, starting from 1.
• k is a smoothing constant, usually set to 60.
• Chunks that do not appear in a given list receive no contribution from that list.

RRF merges rank positions rather than directly adding BM25 scores to vector similarity scores, because the two are on different scales. Chunks that appear near the top of multiple lists receive higher fused scores.

But consistency across retrievers does not prove that a chunk answers the question.

Reciprocal Rank Fusion (RRF) was proposed by Cormack et al. in 2009. It is an unsupervised rank fusion method originally used in metasearch scenarios. Its core advantage is that different retrieval systems produce scores on different scales (for example, BM25 scores are unbounded, while cosine similarity ranges from -1 to 1), and directly adding or weighted-averaging them requires complex score normalization and easily introduces bias; RRF relies only on rank positions, naturally avoiding the score-scale problem. The smoothing constant k in the formula controls how quickly the weight of top-ranked documents decays: the larger k is, the smaller the effect of rank differences and the smoother the fusion result; the smaller k is, the more pronounced the advantage of top-ranked documents. In practice k=60 is widely adopted, but it is not absolutely optimal and needs to be adjusted for the specific task. RRF's limitation is that it assumes every retriever's ranking is reliable and does not account for confidence differences between retrievers, so when one retriever has a large systematic bias, the fused result may be dragged down.

A Simple Example: RRF Can Rank Generic Advice First

Suppose the question is: "How do I fix error AUTH_403?" We have three chunks:

• A: Contains the specific solution.
• B: Contains generic login troubleshooting advice.
• C: Explains password reset.

The rankings are as follows:

Rank	BM25	Vector Search
1	A	B
2	B	C
3	C	A

Using equal-weight RRF and k = 60, chunk B ranks first, followed by A and C. B is near the top of both lists, even though A contains the answer.

This is an illustrative example, not an experimental result. It demonstrates a ranking limitation: A is still retrievable but loses the top position.

RRF merges rankings; it does not independently verify answer relevance.

What I Will Test

I will compare three strategies on the same document collection:

1. BM25 only
2. Vector search only
3. Hybrid search using RRF

The initial pilot will include 15–20 questions, covering exact identifiers, paraphrases, and queries combining both. Before comparing outputs, I will manually label the chunks containing supporting evidence. Each method will use the same corpus, chunking, questions, and labels.

Each individual retriever returns up to 20 candidates. Hybrid retrieval will fuse up to 20 results from each branch—up to 40 unique chunks—and then select the top five.

What I Will Measure

• Hit@5: Is there at least one supporting chunk in the top five?
• MRR@5: How highly is the first supporting chunk ranked? A miss scores zero.
• Retrieval latency: How long does retrieval take, including query embedding time (where applicable)?
• Failure modes: Which questions did each method miss, and why?

I will record whether the hybrid branches run concurrently. Its larger candidate pool also matters when explaining quality and latency differences.

This is a proposed pilot, not completed research. It should reveal useful failure modes, but stronger conclusions require a larger-scale evaluation.

Hit@5 (hit rate@5) and MRR@5 (mean reciprocal rank@5) are two commonly used metrics in information retrieval evaluation. Hit@5 measures whether, for each query, the top 5 returned results contain at least one relevant chunk; it scores 1 if so and 0 otherwise, and is finally averaged over all queries. It reflects the retriever's ability to "find" relevant evidence, but is insensitive to the exact ranking of relevant chunks. MRR@5 further measures how highly relevant chunks are ranked: for each query, if the first relevant chunk is found at rank r in the result list (where r≤5), it contributes 1/r; if no relevant chunk is in the top 5, it contributes 0; the final score is averaged over all queries. MRR@5 gives higher weight to relevant results ranked near the top, so it better reflects "whether the user can see the correct answer in an early position." Used together, the two metrics evaluate both the recall capability and the ranking quality of retrieval.

Key Takeaways

Hybrid search combines lexical matching and semantic similarity. Whether it improves retrieval depends on the documents, the questions, and the setup.

Measure it. Inspect the misses. Confirm whether the retrieved chunks support the answer.

Before blaming the LLM, check the context it received.

Does your RAG system struggle more with exact identifiers or with questions phrased differently?

One-line Summary

Compares BM25, vector search, and RRF hybrid retrieval, noting that hybrid retrieval does not necessarily guarantee answer relevance and emphasizing the need for empirical evaluation.

Paragraph Summary

The article explores the strengths and weaknesses of BM25, vector search, and RRF hybrid retrieval in RAG systems. Vector search excels at semantic matching but may overlook exact identifiers; BM25 is strong at keyword matching but suffers from vocabulary mismatch. RRF fuses the two by merging rank positions, but consistency across retrievers does not guarantee that a chunk answers the specific question, and an example shows generic advice can be ranked ahead of a specific solution. The author proposes a pilot comparing the three strategies on 15–20 questions, evaluated with Hit@5, MRR@5, retrieval latency, and failure modes. The core conclusion: whether hybrid search improves retrieval depends on the specific scenario and must be verified through measurement—before blaming the LLM, first check whether the retrieved evidence supports the answer.