RAG in Production: Should You Really Use Frameworks Like LangChain?

In the AI application development community, the debate over "should we use wrapper frameworks like LangChain/LlamaIndex" never seems to end. A developer who just completed an end-to-end production-grade RAG engine shared his real-world experience on Reddit, offering an answer that takes no side but is highly actionable.

The Core Conclusion: Frameworks Are Tools, Not Religion

The developer's position is clear: if you're just calling chat.completions.create() or doing simple text[:500] text chunking, bringing in LangChain is pure abstraction debt—one more layer of wrapping, one more layer of opaque error stacks.

But when the system moves from a weekend prototype to handling real-world document chaos, the real problems emerge. His judgment: frameworks shouldn't be an all-or-nothing religious belief. The strongest RAG architecture should use native, deterministic Python for heavy data modeling, document parsing, and validation logic, while leveraging mature frameworks for observability, connectors, and evaluation pipelines.

When You Should Write It Yourself

Document AST and Structural Hierarchy

Out-of-the-box chunkers blindly truncate PDFs mid-sentence, or slice a 40-row financial table right down the middle, completely destroying the header context. This developer built a custom AST layout parser using Docling, preserving parent section breadcrumbs, stitching together cross-page tables, and linking tables back to their preceding narrative text.

These kinds of requirements are nearly impossible to implement elegantly in generic frameworks, because the semantic integrity of document structure depends on specific business contexts.

Docling is an open-source document parsing library from IBM, specifically designed for complex formats like PDF, DOCX, and PPTX, capable of outputting a structured document object model (DoclingDocument). It integrates layout analysis models (such as DocLayNet) and table recognition models (such as TableFormer) internally, and can identify heading hierarchies, paragraphs, lists, tables, images, and reading order. Compared to pure text extraction tools like PyPDF2 and pdfplumber, Docling preserves semantic structure information, making it suitable for building custom AST layout parsers. Developers can build on its hierarchical node output to customize parent section breadcrumbs, cross-page table stitching, and the logic for associating tables with narrative text. Docling also supports OCR and multiple export formats (Markdown, HTML, JSON), and is often chosen as the underlying parsing engine in production-grade RAG scenarios that require fine-grained control over document structure.

Domain-Specific Validation Logic

Standard wrappers and generic chunkers won't tell you whether OCR misread a digit or shifted a column. Before retrieval, he implemented sub-2ms character-by-character source text verification against the raw PDF byte layer, as well as arithmetic column reconciliation. This level of validation requires deep understanding of the data itself—generic frameworks simply can't help.

Fine-Grained Retrieval Math

Custom reciprocal rank fusion (RRF) between dense vectors and sparse BM25, combined with context-aware, token-budget-constrained hit folding, is easier to maintain and profile in pure Python.

Reciprocal Rank Fusion (RRF) is a parameter-free hybrid retrieval result merging algorithm, commonly used to fuse ranked lists from dense vector retrieval and sparse keyword retrieval (such as BM25). Its core formula is: for each document, take the reciprocal of its rank in each retriever and sum them with weights, typically using 1/(k + rank) where k is a smoothing constant (often set to 60). RRF's advantage is that it doesn't depend on the absolute scale of scores from different retrievers, using only ranking information, making it robust and easy to implement. In RAG systems, RRF effectively combines semantic similarity with precise keyword matching, improving recall quality. As mentioned in the article, implementing RRF in pure Python makes it easier to embed custom logic like context-aware, token-budget-constrained hit folding, offering more flexibility than the generic fusers built into frameworks.

When You Should Use Frameworks

Observability and Tracing

Integrating LangSmith or OpenTelemetry to trace dynamic tool calls, prompt variants, and latency breakdowns can save hundreds of hours compared to building your own telemetry infrastructure. This is one of the framework's most irreplaceable values.

Ecosystem Portability

Switching vector backends (Qdrant, Chroma, PGVector) or model providers (Mistral, Claude, OpenAI) without refactoring client contracts and middleware logic. This flexibility is especially valuable during the prototyping phase.

Evaluation Standards

Using battle-tested interfaces for benchmark generation fidelity, RAGAS metrics, and citation traceability evaluation. The degree of standardization in the evaluation pipeline directly impacts iteration speed.

RAGAS (Retrieval-Augmented Generation Assessment) is an open-source evaluation framework designed specifically for RAG systems, offering a series of automated metrics that require no human annotation, including: Faithfulness (measuring whether answers are entirely based on retrieved context), Answer Relevance, Context Precision, and Context Recall. These metrics work by calling an LLM to cross-validate generated results against retrieved content, helping developers quantify retrieval quality and generation quality. RAGAS can integrate with observability platforms like LangSmith to form a closed loop from tracing to evaluation. Adopting RAGAS in a standardized evaluation pipeline can significantly accelerate iteration speed and avoid reinventing the wheel—this is also one of the reasons the article emphasizes that frameworks are irreplaceable at the evaluation layer.

Hybrid Architecture Is the Right Answer

The developer's final recommendation is to adopt a hybrid architecture: use native Python for heavy data modeling, document parsing, and validation logic, and use standard frameworks for observability, connectors, and evaluation pipelines.

He has open-sourced the complete technical breakdown and test suite on GitHub, covering AST parsing, dual-format table serialization, hybrid RRF, and LangSmith telemetry. For teams running RAG in production, the core question isn't "whether to use frameworks" but "at which layers to use frameworks."

This pragmatic, layered approach is perhaps more valuable as a reference than either side's extreme position.

One-line Summary

Production-grade RAG should use hybrid architecture: native Python for core logic, frameworks for infrastructure like observability.

Paragraph Summary

Based on a developer's production-grade RAG experience, this article explores whether to use wrapper frameworks like LangChain/LlamaIndex. The core conclusion: frameworks are tools, not religion. Introducing them for simple call scenarios only adds abstraction debt; but for heavy data modeling tasks like document AST parsing, domain-specific validation, and fine-grained retrieval math, native Python is more suitable. Meanwhile, observability (LangSmith/OpenTelemetry), ecosystem portability (switching vector backends or models), and evaluation standards (RAGAS) are where frameworks truly provide irreplaceable value. The final recommendation is a hybrid architecture: native implementation for core logic, mature frameworks for the infrastructure layer. The key question isn't "whether to use" but "at which layers to use."