11-Node LangGraph in Practice: Complete Architecture Breakdown of a Compliance-Grade RAG Agent

As RAG (Retrieval-Augmented Generation) applications become increasingly widespread, a simple "retrieve → generate" pipeline can no longer meet the demands of serious use cases. Recently, a developer shared their Agentic Financial Parser on Reddit's r/LangChain community — an autonomous AI agent built on LangGraph with 11 nodes, specifically designed to parse Indian financial and legal documents (Union Budget, Finance Bills, Income Tax, EPF/EPS pensions, RBI KYC, the Indian Constitution, etc.).

RAG was originally proposed by Facebook AI Research in 2020. Its core idea is to combine the generative capabilities of large language models with external knowledge retrieval, addressing the problems of outdated parametric knowledge and the inability to access private data. The traditional RAG workflow goes: user query → vector retrieval → inject retrieved results as context into the prompt → LLM generates an answer. However, as application scenarios grow more complex, this linear pipeline reveals numerous shortcomings: inability to handle ambiguous queries, lack of post-hoc hallucination verification, and no cost control mechanisms. Agentic RAG is the recent evolutionary direction, introducing agent decision-making capabilities into the RAG pipeline, enabling the system to dynamically select execution paths based on query characteristics rather than running through the entire retrieval process indiscriminately.

The value of this project isn't in being "yet another RAG" — it's in systematically answering a critical question: What engineering guardrails are needed when RAG goes into production environments handling compliance-sensitive data? Intent classification, jailbreak detection, PII masking, ambiguous query clarification, result reranking, hallucination verification, self-correction — all these capabilities are organized into a directed state graph.

From Linear Chains to State Graphs: The Division of Labor Across LangGraph's 11 Nodes

Traditional RAG is a straight line, but this project uses LangGraph's StateGraph to build a graph containing 11 registered nodes. Each node has its own responsibility, working together with conditional routing to achieve complex decision flows.

LangGraph is a framework released by the LangChain team, specifically designed for building stateful, cyclical LLM applications. Unlike LangChain's Chain and Agent abstractions, LangGraph is based on the concept of directed graphs, where each node is a computation step, edges define data flow, and conditional edges enable dynamic routing based on runtime state. Its underlying design draws from Google's Pregel graph computation model, supporting state persistence, human-in-the-loop breakpoints, and subgraph nesting. StateGraph is LangGraph's core class — developers register node functions via add_node, define transition logic via add_edge and add_conditional_edges, and finally compile into an executable graph.

Detailed Core Node Functions

• Classifier: The entry brain of the entire system. Returns structured JSON (intent, document type, confidence) through a single LLM call, then routes to one of 6 paths.
• Reject: Intercepts abusive and jailbreak queries. The key point is that it uses regex blacklists to intercept before the LLM ever sees the content — zero LLM call cost.
• Greet: Handles small-talk queries, completely bypassing the retrieval pipeline and saving vector database overhead.
• CrossQuestioner: Initiates HITL (Human-in-the-Loop) clarification for ambiguous queries, falling back to best-effort retrieval after a maximum of 2 rounds. HITL in AI systems refers to introducing human judgment at critical decision points in automated workflows. LangGraph natively supports this mechanism through interrupt_before or interrupt_after, pausing graph execution at specified nodes, waiting for external input before resuming, with state persisted in the checkpointer during the pause. This "maximum 2 rounds" design embodies the progressive autonomy principle — avoiding infinite clarification loops that degrade user experience.
• Retriever: The heaviest path — a full five-stage RAG pipeline.
• Web Search / Stock Tool: Out-of-scope queries go to Tavily web search; stock queries go to yfinance tool calls.
• Generator: Uses Gemini Flash Lite with temperature 0.1, strictly answering based on context.
• Hallucination Guard: A post-generation answer verification layer.
• Post-Process / Fallback: Persistence, streaming output, and circuit breaker recovery.

Interestingly, PII masking is not a node in the graph but a preprocessing layer running before the graph, using regex matching to mask Aadhaar numbers, PAN card numbers, phone numbers, emails, and bank accounts. This is a critical compliance design detail — sensitive data is sanitized before entering any LLM call.

In the Indian context, the compliance necessity of this design is particularly pronounced: Aadhaar numbers (12-digit unique identity identifiers) are protected under the Aadhaar Act 2016, PAN card numbers (Permanent Account Numbers) are tax-sensitive information, and any unauthorized storage or transmission could violate the DPDP Act (Digital Personal Data Protection Act) passed in 2023. Placing PII masking before LLM calls means: even if API providers promise not to store data, the system architecturally satisfies the "data minimization principle" — never sending unnecessary sensitive data. Regex matching is an efficient masking solution for structured identifiers with strict format specifications (such as Aadhaar's XXXX-XXXX-XXXX format or PAN's 5-letter-4-digit-1-letter format).

6-Way Intelligent Routing: Classifier-Driven Decisions

One of the most elegant designs in this architecture is the classifier returning one of 6 routes, implemented via add_conditional_edges:

• reject: Abuse/jailbreak
• greet: Greetings/small talk
• cross_question: Ambiguous query → HITL clarification
• web_search: Out-of-scope → Tavily
• stock_tool: Stock queries → yfinance
• retriever: Legal/financial → Full RAG

The benefits of this design are obvious: not all queries deserve the full retrieval pipeline. Greetings and out-of-scope queries are diverted early, saving both vector database and LLM costs while reducing end-to-end latency. This embodies the core philosophy of cost-sensitive agents — replacing expensive execution with cheap judgment.

RAG Retrieval Pipeline: Five-Stage Precision Engineering

As the heaviest path, the retriever contains five sequential stages, each with clear engineering trade-offs:

The Delicate Balance Between Storage and Recall

1. Jina AI v3 MRL Embeddings: Embedded at 1024 dimensions then truncated to 256 dimensions. This saves 75% of Pinecone storage space with negligible quality loss. This leverages MRL (Matryoshka Representation Learning) technology, allowing flexible use of different dimensions from the same embedding. MRL was proposed by University of Maryland researchers in 2022, named after Russian nesting dolls — the first d dimensions of an embedding vector are themselves a valid d-dimensional representation. Traditional embedding models produce fixed-dimension vectors, and dimensionality reduction through post-processing methods like PCA incurs significant loss. MRL simultaneously applies loss functions to multiple prefix dimensions (e.g., 32, 64, 128, 256, 512, 1024) during training, teaching the model to encode the most important information in the earlier dimensions. Therefore, when truncating from 1024 to 256 dimensions, key semantic information is preserved while storage and retrieval computation costs are dramatically reduced.

2. Pinecone Serverless: Dual-namespace design with 14,662 active vectors.

3. Parent-Child Chunk Parsing: First retrieve small child chunks (high precision), then fetch parent chunks from Supabase (high context density). This is the classic solution to the contradiction of "small chunks are precise but lack context; large chunks are complete but noisy." Specifically, this strategy addresses the "retrieval granularity paradox" in RAG: small text chunks (e.g., 100-200 tokens) achieve high precision in semantic retrieval because they match queries more precisely; but they lack sufficient context for LLM comprehension and reasoning. Large text chunks (e.g., 1000+ tokens) contain complete context but introduce substantial irrelevant noise, reducing retrieval hit rates. The parent-child chunk solution: index with small chunks for computing embeddings and similarity matching, but substitute them with their corresponding large chunks in the actual context sent to the LLM. Implementation requires a relational store (like Supabase/PostgreSQL in this project) to maintain child-to-parent chunk mappings.

4. Cohere Rerank v3.0: 15 candidates reranked to Top 10 "golden chunks," significantly improving multi-document query quality. Reranking is a two-stage strategy in information retrieval: the first stage uses efficient but relatively coarse vector similarity to recall many candidates; the second stage uses a more precise but computationally expensive Cross-Encoder model to precisely rank the candidates. Cohere Rerank v3.0 concatenates the query with each candidate document and feeds them into a Transformer, directly outputting relevance scores that capture fine-grained query-document interactions. Reranking from 15 candidates to Top 10 means approximately 33% of initial candidates are filtered, effectively improving the signal-to-noise ratio of context sent to the LLM.

5. Confidence Gating: Graceful degradation when scores are <30%, HITL prompt triggered when <45%. This is the critical design for preventing hallucinations at the source.

Hallucination Guard Design: Advisory Rather Than Blocking

One thoughtfully considered design decision in the project: the hallucination guard adopts an advisory mode rather than a blocking mode.

After generating an answer, the system uses an independent LLM call for "LLM-as-Judge" verification: "Is this answer based on the provided context? Answer YES or NO." If judged as not grounded, the system appends a disclaimer but still returns the answer.

LLM-as-Judge is an important paradigm in LLM application evaluation in recent years, first applied at scale in Stanford's Alpaca evaluation and LMSYS's Chatbot Arena. Its core idea is using one LLM to judge the output quality of another LLM, replacing expensive human annotation. In RAG hallucination detection scenarios, the Judge model's task is to determine whether the generated answer is "grounded" in the provided retrieval context. Academic frameworks like RAGAS and TruLens have standardized this paradigm as one of the core metrics for RAG evaluation: Faithfulness. However, this approach has inherent limitations: the Judge model itself can make errors, and it adds an extra LLM call's latency and cost — which is one reason this project chose advisory mode over blocking mode.

The rationale behind this trade-off deserves deep consideration: outright blocking creates a poor user experience when the LLM legitimately knows things beyond the retrieval scope. The disclaimer hands the trust judgment back to the user. This is highly valuable in production environments — overly strict guardrails often backfire.

Production-Grade Resilience: Circuit Breakers and HITL Permission Control

Circuit Breakers Prevent Cascading Failures

Both LLM and embedding APIs are wrapped with pybreaker: 3 consecutive failures → circuit opens → instant fallback for 30 seconds → retry after half-open. This prevents request piling and cascading failures — an essential engineering practice for deploying agents into real traffic.

The Circuit Breaker Pattern originates from the overload protection concept in electrical engineering, introduced to software engineering by Michael Nygard in his 2007 book Release It!, and later widely popularized by Netflix's Hystrix library. Its core state machine contains three states: closed (requests pass normally), open (all requests rejected with immediate degraded response), and half-open (a small number of probe requests allowed through to detect service recovery). In LLM applications, circuit breakers are particularly important: LLM API latency is high (typically 1-10 seconds) and billing is per-token. Without a circuit breaker, requests pile up during intermittent failures, causing users to wait for extended periods, with retries after timeouts generating even more wasted overhead. This project's settings of a 3-failure threshold and 30-second recovery window are a reasonable match for LLM API failure modes (typically brief rate limits or service overload).

Permission Flow Design for Web Search

The system never automatically triggers Tavily. When retrieval confidence is low, it first asks the user: "I couldn't find this in the documents — would you like me to search the web?" Only after user consent does it route to the web search node. This both controls API costs and gives users control over the agent "leaving its knowledge boundary." This design embodies the "progressive autonomy" principle — the agent acts autonomously within knowledge boundaries, seeks human permission at boundaries, avoiding unpredictable behavior and cost overruns that fully autonomous systems might cause.

Zero-Cost Infrastructure and Performance Metrics

Perhaps most impressively, the entire system runs on various free tiers: Render (512MB) deployment, Pinecone Serverless, MongoDB Atlas (30-day TTL), Supabase, Upstash Redis (<100ms semantic cache), Gemini Flash Lite, Langfuse observability, Cohere, and Jina.

Key Performance Data

• graph.py contains 1,809 lines of code
• 11 registered nodes, 6 routing paths, 2 circuit breakers
• 14,662 active vectors indexing 20+ Indian government acts
• Cache hits <100ms, cold-start end-to-end latency <8 seconds

Additionally, the same 11-node pipeline is served externally via the Meta WhatsApp Cloud API — the same graph, the same guardrails, the same PII shield, with no separate bot logic needed.

Conclusion: A Design Decision Checklist for RAG Engineering

The significance of this project lies in integrating scattered "best practices" into a runnable, observable, low-cost deployable system. For any team planning to deploy RAG in compliance-sensitive scenarios, it provides at least several questions worth considering:

• Should the hallucination guard be blocking or advisory?
• How to balance the quality/storage trade-off of MRL embedding truncation?
• What circuit breaker threshold to set for LLM APIs?
• Should the agent auto-search or ask permission first?

Production-grade Agentic RAG is far from having standard answers — behind every design decision lies a multi-dimensional trade-off between cost, experience, security, and accuracy. The project is open-sourced on GitHub for developers interested in diving deeper into its implementation details.