Open-Source Document Intelligence in Practice: RAG-Based PDF Q&A with Multi-Model Switching

A developer shared an open-source document intelligence project on Reddit, built around a RAG (Retrieval-Augmented Generation) Q&A system for PDF, DOCX, and other document formats. Rather than aiming for a polished, production-ready product, the author is primarily seeking community feedback on the chunking and retrieval strategy, as well as the multi-model switching design. For developers interested in RAG architecture and local document Q&A, this project offers a complete, reproducible implementation to learn from.

What the System Does

The workflow is straightforward: users upload PDF, DOCX, or TXT files, and the system automatically chunks the content, vectorizes it, and stores the embeddings in a Postgres + pgvector setup. From there, users can ask natural-language questions about the document content.

One design decision worth highlighting is the vendor-agnostic model layer. The author uses a single environment variable to switch between OpenAI and Claude as the underlying LLM, avoiding lock-in to any single provider. Given today's rapidly evolving model landscape — where capabilities shift quickly and pricing fluctuates — this is a pragmatic engineering choice. When a model's cost-performance ratio drops or a better option emerges, the migration cost is minimized.

Tech Stack Breakdown

The project uses a mainstream, well-established full-stack combination:

• Frontend: React + TypeScript
• Backend: Node.js / Express
• RAG Orchestration: LangChain
• Vector Storage: Postgres + pgvector
• Local Development: Docker Compose

The stack prioritizes engineering maintainability over novelty. Choosing Postgres with pgvector instead of a dedicated vector database (like Pinecone or Milvus) means teams can reuse existing relational database operational knowledge, while keeping structured and vector data in the same storage layer — reducing architectural complexity. For small-to-medium scale document retrieval, this is a cost-effective and practical choice.

LangChain serves as the RAG orchestration layer, wiring together the full pipeline: document loading, chunking, embedding, retrieval, and generation — while also providing the abstraction needed for multi-model switching.

About pgvector: pgvector is an open-source PostgreSQL extension that natively adds vector data types and similarity search capabilities to a relational database, supporting both exact nearest neighbor (KNN) and approximate nearest neighbor (HNSW, IVFFlat index) retrieval modes. Compared to dedicated vector databases like Pinecone or Weaviate, pgvector's key advantage is zero additional infrastructure — vector data and business data coexist in the same Postgres instance, with transactions, permissions, and backup strategies all reusable. The trade-off is that at very large scale (tens of millions of vectors and above), its index build speed and query throughput typically lag behind dedicated solutions. For internal knowledge bases with up to tens of thousands of documents, pgvector's performance is more than sufficient — a textbook example of the "good enough is good enough" engineering philosophy.

The Value of Multi-Model Switching

In real-world RAG deployments, vendor lock-in is a genuine pain point. Different LLMs have varying strengths in long-context understanding, structured output, and cost efficiency, and business requirements shift constantly. The author's approach of swapping between OpenAI and Claude via a single configuration item essentially treats the model as a pluggable component.

The benefits go beyond just switching models: this design inherently requires developers to decouple prompt construction and retrieval logic from specific model APIs. Once that abstraction is clean, integrating additional models — such as the Llama family or other open-source alternatives — becomes much smoother. The author specifically calls out this provider-swap design as an area where they're seeking community feedback, signaling awareness that the abstraction layer's boundaries and robustness still have room for refinement.

Chunking and Retrieval: The Make-or-Break of RAG

The author openly states that chunking and retrieval are the areas where they most want feedback — and that hits on exactly what determines RAG system quality.

Document Q&A quality is highly dependent on chunking strategy: chunks that are too large dilute key content with irrelevant noise; chunks that are too small break semantic continuity and lose contextual relationships. For PDFs with complex layouts, tables, and multi-column formatting, the challenge is even greater. Retrieval is equally critical — pure vector similarity search often struggles with exact keyword matching and multi-hop reasoning. Hybrid retrieval (vector + keyword) and reranking are frequently essential for improving recall quality.

By exposing these challenges in runnable code, the project actually becomes excellent study material for understanding RAG engineering trade-offs: readers can examine a real implementation, understand the decisions made at each step, and move beyond purely conceptual understanding.

Chunking Strategy Overview: The main approaches fall into three categories: fixed-size chunking (hard cuts by character or token count), recursive character chunking (LangChain's default, which prioritizes semantic boundaries by splitting paragraph → sentence → word), and semantic chunking (using embedding models to compute similarity between adjacent sentences and splitting at sharp similarity drops). Fixed-size chunking is simple to implement but risks semantic fragmentation; recursive character chunking performs reliably in most general scenarios; semantic chunking produces higher quality but at greater computational cost, with limited benefit for short documents. For PDF-specific challenges, additional approaches include hierarchical chunking based on document structure (heading levels, paragraph tags), and multimodal chunking for handling tables and charts. On the retrieval side, hybrid retrieval typically combines Dense Retrieval with sparse keyword retrieval (BM25) using Reciprocal Rank Fusion (RRF), followed by Cross-Encoder reranking — significantly compensating for pure vector retrieval's shortcomings in exact-match scenarios.

How to Approach This Kind of Reference Implementation

The author is clear that this is "a runnable reference implementation, not a polished, production-deployable product." This is an honest positioning, and it also sets the right expectations for users: treat it as a starting point for learning RAG architecture and validating technology choices, not something to ship directly to production.

For developers looking to get started with document intelligence or build an internal knowledge base, the value of open-source projects like this lies in providing a complete, reproducible technical path — from document parsing to vector storage to multi-model Q&A, with every step traceable. Before real production use, substantial work remains on chunking precision, retrieval accuracy, access control and security, and performance at scale.

The project is open-sourced on GitHub (Srameshgitnow/AI-DocumentIntelligence) — interested developers can browse the source code and join the discussion directly.