Hierarchy-Aware RAG Chunking Toolkit: A Private Deployment Solution Built for Legal Documents

Why RAG Chunking Deserves Its Own Dedicated Tool

In a RAG (Retrieval-Augmented Generation) pipeline, chunking is often the least glamorous yet most impactful step. Chunk too small, and context breaks apart; chunk too large, and retrieval noise increases. The trickier challenge comes with highly structured documents like legal briefs, contracts, and research papers — they naturally carry a multi-level nested structure of "title → subtitle → section → subsection," yet most general-purpose chunkers simply split by fixed character count or paragraph breaks, discarding all that structural information in the process.

A developer posted a toolkit on Reddit called the RAG Chunking Processing Bundle, centered on a "Hierarchy-Aware Chunker" along with two cross-reference extraction scripts tailored for legal documents. What makes it most noteworthy isn't just the functionality itself, but the shift in how it's delivered — from an online service to a fully self-hostable source code package.

The basic mechanics of RAG (Retrieval-Augmented Generation): external documents are split into chunks, stored in a vector database, and when a user asks a question, the most relevant chunks are retrieved and fed as context to a large language model to generate an answer. Chunk quality directly determines retrieval recall — if a complete argument is split across two chunks, vector search may only retrieve half the content, causing the LLM to answer based on incomplete information. Common approaches like fixed-size chunking and paragraph-based chunking are simple to implement but limited for richly structured documents (legal statutes, academic papers, technical specifications), because they completely flatten hierarchical relationships like "Chapter 3, Section 2, Article 1," leaving the LLM with no way to determine a chunk's structural position during retrieval.

From SaaS to Source Code Delivery: Privacy Is the Core Demand

The author had previously built this chunker as a third-party service, but after talking with users, discovered a recurring need: many teams don't want to send their documents to an external service. They want absolute privacy, on-premise deployment, full control over their infrastructure, and no vendor lock-in.

This demand is especially intense in sensitive industries like law, finance, and healthcare. Contracts, case law, and regulatory documents often carry confidentiality obligations, and any data flowing through a third-party API can create compliance risk. So the author repackaged the entire document processing toolkit as a complete Python package with full source code, letting users integrate it directly into their own projects and run it in air-gapped environments. This "sell the code, not the service" approach hits squarely at the real pain points of enterprise RAG deployment.

Vendor lock-in refers to a situation where an organization becomes so deeply dependent on a particular vendor's API or service that migrating to an alternative becomes difficult, eroding their negotiating leverage. For RAG pipelines, this risk is particularly concrete: if chunking logic, vector indexing, and retrieval strategy are all hosted on an external platform, any pricing change, API modification, or service discontinuation forces a full rebuild of the production system. On-premise deployment keeps all computation within the organization's own or self-controlled infrastructure — data never traverses the public internet, eliminating both compliance risk and external dependencies. As data protection regulations like GDPR and HIPAA tighten, this deployment model is becoming the de facto standard — not an optional choice — in finance, legal, and healthcare industries.

Hierarchy-Aware Chunker: Every Chunk Knows Which Section It Belongs To

The core capability of this chunker is understanding document structure — recognizing headings, heading levels, sections and subsections, and merging nested subheadings into the correct chunk to ensure context flows coherently.

Several key features are worth breaking down:

• Preserves multi-level hierarchy: Supports multi-level nesting across Title → Subtitle → Section → Subsections, rather than flattening everything into a single layer.
• Attaches metadata to every chunk: Each chunk knows which section it belongs to, so the retriever and LLM always have clear context about structural position.
• Retrieval-friendly structured output: Generated chunks are context-aware, structured, and optimized for retrieval.

The example output the author provided makes the effect immediately clear. For a document titled Magistrates' Courts (Licensing) Rules (Northern Ireland) 1997, the chunked result includes not just the body text but also the full hierarchical path in the Metadata:

--- Chunk 2 ---
Metadata:
  Title: Magistrates' Courts (Licensing) Rules (Northern Ireland) 1997
  Section Header (1): PART I
  Section Header (1.1): Citation and commencement
Page Content:
  PART I Citation and commencement
  1. These Rules may be cited as...

Headings are preserved and attached to each chunk, so the retriever and LLM always know which section and subsection a given fragment belongs to. The author emphasizes that this means you no longer need to constantly tune chunk overlap and chunk size — for anyone who has done RAG optimization, this is a genuinely time-saving selling point.

Cost and Integration: LLM Usage on Demand

The tool doesn't blindly rely on large language models. It combines LLM inference with a proprietary optimized parser to keep costs in check. The author specifically notes: if OCR can perfectly identify headings and subheadings, no LLM calls are needed at all. It's also compatible with any provider in the LangChain ecosystem (OpenAI, Anthropic, Google, Ollama), making it easy to swap providers. No additional preprocessing is required — just paste raw content or Markdown directly.

Two Legal-Specific Scripts: Citation Extraction Without LLMs

The toolkit also includes two extraction scripts tailored to legal documents, both built around pure rule-based parsing with no LLM dependency — meaning faster, cheaper, and better suited to privacy-sensitive environments.

Legal Cross-Reference Extractor

This script extracts structured legal citations from an entire document, covering Sections, Articles, Rules, Paragraphs, Schedules, Clauses, Regulations, Orders, and more — including complex compound references. From the example output, it can correctly parse highly nested and range-style expressions like Article 63(9)(b)(iii), Articles 25, 26, 26A, 26D, Articles 73 to 79, and Order 6, rule 10.

Legal citation formats are extraordinarily varied and irregular. Reliably handling these compound references through rule-based parsing alone is a significant engineering challenge. For teams building legal knowledge graphs or statute-linkage retrieval systems, this kind of structured citation data is an essential requirement.

Cross-references in legal documents refer to instances where provisions within a document — or across documents — cite each other, such as "pursuant to Article 63(9)(b)(iii)...". These citations appear at high density in legislative text and form a semantic dependency network among provisions. For RAG systems, if these citation relationships aren't extracted and explicitly modeled, the retriever can typically only recall the directly queried provision and cannot automatically surface the upstream provisions it references — leaving answers without a complete legal chain of reasoning. Once citation relationships are structured, they can be used to build knowledge graphs that support queries like "all provisions that cite this article" or "the full citation path for this Act," significantly deepening the reasoning capability of legal AI applications.

Legal Act Extractor

The second script is more focused: it automatically extracts the names of all Acts referenced in a document, such as Family Law Act 1975, Legislation Act 2003, and Taxation Administration Act 1953. Used alongside the cross-reference extractor, it provides a fast way to map the external dependencies of any legal document.

Use Cases and a Realistic Assessment

As the author positions it, this toolkit is particularly well-suited for RAG processing of structured documents like legal briefs, research papers, and contracts. The author also provides an online Playground at hierarchychunker.codeaxion.com for testing, as well as a YouTube demo video using real documents.

It's worth being clear-eyed: this is a self-reported writeup with commercial promotion from a Reddit post. The example results and performance claims are provided entirely by the author, with no third-party benchmarks or independent verification. Whether the two legal scripts' rule coverage generalizes across different legal jurisdictions and document formats remains to be tested in practice. Teams considering adoption are advised to first validate chunking quality on their own real documents in the Playground before committing to a production pipeline.

That said, the trend it reflects is clear: as RAG matures from demo projects to enterprise deployment, data privacy and on-premise hosting are becoming hard requirements, and domain-specific structured preprocessing — particularly for fields like law — is emerging as a key differentiator for retrieval quality.