Corrective RAG (CRAG) in Practice: Building a Compliance Research Assistant That Self-Validates

The Fatal Flaw of Ordinary RAG: Confidently Wrong Answers

Most RAG chatbots share a common problem — they confidently dump whatever they've retrieved straight to the user, even when the content is irrelevant or outright wrong. In casual conversation, this might be a minor annoyance. But in compliance, a single bad retrieval result can quickly escalate into real regulatory risk and legal exposure.

Developer Saurabh Kamal built a compliance assistant called Veltra Pay Compliance Assistant to tackle this exact pain point. The core idea is to introduce Corrective RAG (CRAG) and a self-correction mechanism, forcing the system to rigorously audit its own work before handing any answer to the user.

What Is Corrective RAG (CRAG)?

Traditional RAG follows a linear pipeline: retrieve documents → assemble context → generate answer. The biggest vulnerability in this chain is the implicit assumption that "whatever was retrieved must be useful." The moment vector search pulls back an irrelevant chunk, the model will typically still push through and produce a plausible-sounding answer — which is exactly how hallucinations are born.

CRAG addresses this by inserting a "quality checkpoint" into the retrieval step. The system uses an LLM to score each retrieved document, determining whether it is relevant or irrelevant. When retrieval quality is weak and there aren't enough relevant documents, the system doesn't settle for a shaky answer — it proactively rewrites the search query, rephrases the question, and retrieves again until it has gathered sufficiently reliable evidence.

This "evaluate first, then decide" approach transforms RAG from a passive pipeline into an active system capable of judging information quality.

The concept of CRAG was formally introduced by Yan et al. in the 2024 paper Corrective Retrieval Augmented Generation. Its core innovation is a lightweight "retrieval evaluator" that quantifies the relevance of each retrieved document as a confidence score, then routes documents into one of three handling paths: high scores are used directly, low scores are discarded entirely and trigger a web search, and mid-range scores go through knowledge refinement before use. This stands in sharp contrast to the passive "use whatever you have" approach of traditional RAG. Notably, query rewriting in CRAG isn't simply rephrasing the question for another search — the LLM decomposes the original query into several more precise sub-queries, retrieves for each separately, then merges the results. This significantly reduces retrieval failures caused by vocabulary mismatch.

The Self-Correcting Agent: One More Validation Pass After Generation

If CRAG solves the problem of "is the retrieval good enough?", the project's second layer — the Self-Correcting RAG Agent — solves the problem of "is the generated answer actually reliable?"

After the answer is generated, the Agent validates its own output across the following dimensions:

• Grounding: Is the answer genuinely built on the retrieved source documents, rather than the model's own speculation?
• Hallucination risk: Does the output contain any claims the source material cannot support?
• Completeness: Does the answer fully address the user's question?
• Retrieval sufficiency: Is the available evidence strong enough to justify this response?

What matters most is the behavioral logic: when validation fails, the system automatically retries. And when it genuinely cannot find an answer in the source material, it honestly responds with "I don't know" rather than fabricating something. In compliance contexts, this kind of restraint — choosing silence over a wrong answer — is often more valuable than generating more output.

Tech Stack Breakdown: LangGraph Powers the Controlled Retry Loop

From an engineering perspective, the project's technology choices are highly representative of modern Agent application patterns:

• LangGraph: Serves as the orchestration core for the entire pipeline, using StateGraph to define nodes, conditional edges, and bounded retry loops. Bounded retries are especially important — they prevent the Agent from getting stuck in infinite loops by setting a clear ceiling on self-correction attempts.
• Qdrant: The vector database responsible for storing and retrieving document embeddings.
• PDF ingestion and chunking: PDF parsing via pdfplumber, with an overlapping chunks strategy to preserve contextual continuity across boundaries.
• LLM-driven core components: Document scoring, query rewriting, and answer validation are all handled by a large language model.
• Streaming backend: Pipeline progress is pushed to the frontend in real time via Server-Sent Events (SSE).
• Web dashboard: Provides real-time visualization of the entire CRAG pipeline.

The combination of SSE streaming and the visualization dashboard is worth highlighting. It turns what would otherwise be a "black box" RAG process — when is it scoring, when did a rewrite get triggered, which step failed and retried — into something fully observable. For compliance use cases that demand auditability and explainability, this kind of transparency is a first-class requirement, not a nice-to-have.

LangGraph is a graph-structured Agent orchestration framework from the LangChain team. Its key difference from traditional chain-based orchestration is that it supports stateful cycles. In a standard LangChain pipeline, execution flows as a one-directional DAG (directed acyclic graph), making it impossible to natively express logic like "on failure, return to a node and retry." LangGraph's StateGraph allows conditional back-edges between nodes, which is exactly what makes self-correcting retry loops possible. "Bounded retries" means maintaining a retry counter in the StateGraph state object, incrementing it each time the system enters a retry node, and when the count exceeds a preset threshold (e.g., 3 attempts), the conditional edge routes the flow to a terminal node that returns "I don't know" rather than continuing to loop. This balances fault tolerance against infinite recursion — a design pattern with broad applicability to any Agent application requiring adaptive retries.

Qdrant supports switching between in-memory and persistent modes, enabling smooth progression from prototype to production. The overlapping chunks strategy typically uses 10–20% overlap, ensuring that semantic units spanning chunk boundaries (such as a complete regulatory clause) aren't split across two chunks in a way that leaves either chunk semantically incomplete — which would otherwise degrade vector retrieval accuracy.

What This Means for Developers

The value of this project lies not just in being a runnable compliance assistant, but in demonstrating how to treat reliability as a first-class citizen when designing RAG systems. Ordinary RAG optimizes for "can it answer?" — CRAG and the self-correcting Agent optimize for "can it answer responsibly?"

For any team deploying RAG in high-stakes domains like finance, healthcare, or law, this combination of document scoring, query rewriting, answer validation, bounded retries, and the willingness to honestly say "I don't know" is well worth adopting as an engineering template. The author has open-sourced the full code on GitHub, with an accompanying YouTube walkthrough — developers who are interested can reproduce the entire pipeline from there.

Note: This article is based on a single source — the developer's share on Reddit. For specific performance and effectiveness, we recommend verifying directly with the open-source repository and video.