Staged RAG Architecture in Practice: Making Retrieval Failures Explicit and Controllable

Rethinking RAG Pipeline Design

Traditional RAG (Retrieval-Augmented Generation) tends to treat retrieval as a black box: run a vector query, grab the Top-K results, and dump them straight into the context for the LLM. It's simple, but it hides a serious risk — when retrieval quality is poor, the model is forced to generate content based on weak or outright wrong context, producing hallucinations with no warning whatsoever.

A developer shared an experimental RAG architecture on Reddit, built around a core idea: break retrieval, reranking, filtering, and context construction into distinct, independent stages instead of compressing everything into a single retrieval step. The key value of this design is making retrieval failures explicit.

The Complete Staged Pipeline

The project (open-sourced on GitHub as mini-Rag) implements a processing chain with clearly separated responsibilities:

BGE-M3 → Dense + Sparse Embeddings → Qdrant
→ Hybrid Retrieval / RRF → BGE Reranker
→ Relevance Gate → Context Quality Filter
→ Context Builder → Local LLM

Each stage handles a single responsibility. Let's break down the key components.

Hybrid Retrieval and RRF Fusion

The pipeline starts with the BGE-M3 model, which generates both dense and sparse embeddings stored in the Qdrant vector database. Dense vectors excel at capturing semantic similarity, while sparse vectors preserve exact keyword matching capabilities.

Hybrid Retrieval combined with RRF (Reciprocal Rank Fusion) merges results from both approaches, balancing semantic understanding with literal keyword hits. In practice, this combination tends to be more robust than pure dense retrieval — especially when dealing with proper nouns, code snippets, or precise technical terms.

RRF is a rank fusion algorithm that requires no additional parameter training. Its core formula works as follows: for each candidate document, sum the reciprocals of its rankings across all rank lists (typically with a smoothing constant k, defaulted to 60). A higher combined score means a better overall ranking. The algorithm's strength lies in its insensitivity to extreme rankings — a document ranked 1st scores roughly 1/61, while one ranked 10th scores about 1/71, making the score differences relatively gradual. This prevents any single retrieval method's unusually high scores from dominating the final result. Compared to weighted averaging of raw similarity scores, RRF requires no calibration across different retrieval methods' score scales, making it straightforward to implement and highly robust. This property is especially valuable when dense and sparse vector retrieval produce score distributions with very different characteristics.

Reranking and the Relevance Gate

Initial retrieval results go through a BGE Reranker for reordering, using a more precise cross-encoder model to rescore the candidate passages. After reranking comes the most distinctive element of this architecture — the Relevance Gate.

This is where the author's core philosophy shines: if the reranker fails to find sufficiently relevant passages, the system returns an "insufficient information" response rather than forcing weak context onto the model. Think of it as a safety valve for RAG — it would rather honestly say "I don't know" than fabricate a plausible-sounding but invented answer.

The cross-encoder used by BGE Reranker differs architecturally from the bi-encoder used in the initial retrieval stage. Bi-encoders independently encode the query and document before computing similarity — fast, but limited in precision. Cross-encoders concatenate the query and document and process them together, allowing full attention-layer interaction between the two for more accurate scoring, at higher computational cost. This precision difference is exactly why a reranker is suited for a second-pass ranking after initial retrieval narrows down the candidate set, rather than being applied directly to the full document corpus. The relevance gate threshold is typically set based on the reranker's normalized output scores, requiring careful balancing between precision and recall — a key engineering challenge the author highlights in the post.

Why "Explicit Failure" Matters So Much

In production environments, the most dangerous RAG failure mode isn't "can't answer" — it's "confidently answers incorrectly based on wrong context." The latter is far more misleading to users than the former.

Making retrieval failures explicit means:

• Better observability: Gate and quality filter decisions can be logged and monitored, helping identify gaps in knowledge base coverage;
• Increased trustworthiness: User-facing systems can clearly distinguish between "answering from documents" and "unable to answer," preventing hallucination contamination;
• Clearer accountability: Once each stage is independent, debugging isn't a guessing game against a black box — you can pinpoint whether the issue lies in retrieval, reranking, or context construction.

The Context Quality Filter and Context Builder further ensure the quality of what ultimately gets fed to the LLM — the former removes low-quality passages, while the latter handles structuring the final context.

The Case for Local Deployment

On the generation side, this architecture uses DeepSeek^Ad R1 7B, running locally via Ollama. This choice reflects a deliberate consideration for privacy and cost: the entire pipeline — from embeddings and vector store to the LLM itself — can run in a closed loop locally, with no dependency on external APIs.

For internal enterprise knowledge bases, code retrieval, or data-sensitive scenarios, this fully local setup has obvious appeal. A 7B-scale model paired with high-quality context filtering also partially compensates for smaller models' weaker reasoning capabilities — after all, the higher the context quality, the less room there is for the model to hallucinate.

Ollama is an open-source tool designed for running large language models locally. By wrapping the llama.cpp inference engine, it lets users pull and run quantized open-source models (like LLaMA, Mistral, DeepSeek, etc.) with a single command, without manually handling model weight format conversions or low-level inference configuration. For RAG use cases, Ollama also provides a local endpoint compatible with the OpenAI API format, allowing existing API-based code to switch to local inference with almost no changes. DeepSeek R1 7B is a reasoning-enhanced small model, specifically trained on mathematical and logical reasoning tasks. It makes relatively efficient use of structured context given its parameter constraints — a good fit for the strict context quality filtering that staged RAG enforces.

Open Questions About Architectural Boundaries

The author raises an open question in the post — a classic challenge in RAG engineering: Where exactly should the boundaries be drawn between retrieval → reranking → context preparation → generation?

There's no standard answer. Over-segmentation increases latency and system complexity; under-segmentation makes failure modes opaque. The relevance gate threshold is especially tricky — set it too high and you'll frequently trigger "insufficient information" responses, hurting usability; set it too low and the safety valve loses its purpose.

For teams building production-grade RAG systems, this staged design offers a clear reference framework: decompose the "single retrieval" black box into a series of independent stages — each measurable, tunable, and capable of failing explicitly. Controllable retrieval quality often determines a RAG system's reliability more than simply throwing a more powerful model at the problem.