PDF to Markdown: Fixing Reading Order and Tables Before RAG Chunking

Title

PDF to Markdown: Fixing Reading Order and Tables Before RAG Chunking

Meta Description

RAG giving inaccurate answers to PDF questions? The problem often lies in the extraction stage. This article analyzes how two-column reading order, table structure, and heading hierarchy get destroyed by plain text extraction, compares open-source libraries like pypdf, pdfplumber, Docling, and Marker, and provides a runnable PDF-to-Markdown Python script to fix chunking quality at the source.

Article Content

When a RAG pipeline performs poorly on PDF questions, most people's first instinct is to tune embeddings or adjust prompts—but the problem often lies in the extraction stage. Two-column pages get scanned line by line, tables become a string of numbers, and headings are reduced to ordinary text lines. Before you've even started chunking, you're spending an afternoon on manual cleanup. This article analyzes the root causes of these problems, explains why Markdown (rather than plain text) solves most of them, compares mainstream open-source libraries, and provides a locally runnable Python script so you can see the difference for yourself.

Three Things Plain Text Extraction Breaks

Reading order in two-column layouts. Most extractors work line by line, top to bottom. In a two-column paper, the first line of the left column is followed by the first line of the right column, so every chunk mixes two unrelated passages. The embedding lands somewhere between the two topics, and retrieval accuracy suffers.

Table structure. When a table is flattened into text, the values remain but the structure is lost. "Q1 Q2 Q3 / 12 15 9" can't tell the model which number belongs to which column. A Markdown table (| Q1 | Q2 | Q3 |) binds the header to each row, and LLMs are very good at reading this format.

Heading hierarchy. Headings are the best chunk boundaries and useful metadata (e.g., "this chunk comes from 3.2 Limitations"). But in plain text, they're indistinguishable from any other line. In Markdown, headings are #/## lines, so LangChain's MarkdownHeaderTextSplitter or a ten-line function can split by heading.

CJK documents have additional problems: justified Chinese text often inserts spaces between characters, and line breaks fall mid-sentence—both interfere with keyword search and chunk size calculations.

Choosing an Open-Source Library: From Lightweight Rules to ML Pipelines

According to PyPI and GitHub metadata, the following projects are actively maintained; verify licenses yourself:

Library	Approach	Strengths	Caveats	License
pypdf	Pure Python text extraction	Zero dependencies, fast	No layout, tables, or headings	BSD-3
pdfminer.six	Layout analysis (chars→lines→boxes)	Fine-grained positions, fonts, sizes	You build the structure yourself	MIT
pdfplumber	Built on pdfminer.six	Bordered table detection, easy word/char access	Borderless tables need tuning	MIT
PyMuPDF / pymupdf4llm	MuPDF engine, outputs Markdown	Fast, Markdown ready out of the box	AGPL-3.0 or commercial license	AGPL/commercial
Docling (IBM)	ML layout + table models	Complex layouts, multiple formats	Heavy install, requires model downloads	MIT
Marker	ML pipeline, optional LLM	Difficult layouts, formulas	GPU recommended; check model weight terms	Apache-2.0 (code)
MinerU	ML pipeline, OCR	Scanned documents, Chinese	Heavy; custom license	MinerU license
unstructured	Element partitioning	Multiple file types, element metadata	High-res mode pulls in many dependencies	Apache-2.0
MarkItDown (Microsoft)	Multi-format to Markdown	Office files, fast conversion	Light PDF handling (uses pdfminer/pdfplumber under the hood)	MIT

A rough rule: if the PDF has a text layer (native digital reports, papers, government documents), a layout-aware rule-based approach built on pdfminer.six/pdfplumber is lightweight, deterministic, and cheap to run. If it's a scanned document or highly visual, you need OCR or an ML pipeline (Docling, Marker, MinerU), at the cost of compute overhead.

The text layer is the prerequisite for deciding on a PDF processing route. Native digital PDFs write each character's encoding, position, font, and size into the content stream during generation, forming a text layer that programs can read. Scanned documents are essentially images with no such layer, so extract_words() in libraries like pdfplumber returns an empty list. A quick test: open a page with pdfplumber and call page.extract_text(). If it returns an empty string or many (cid:NN) tokens, the text layer is missing or the font mapping is broken, and you should switch to an OCR/ML pipeline. Text layer quality is also affected by font embedding—subsetted fonts or custom encodings can lose the character mapping table, producing garbled extraction. This is also the root cause of the "symbol font" limitation mentioned in the article.

XY-cut is a classic document layout analysis algorithm: project text blocks onto the X-axis (horizontal) and Y-axis (vertical), find whitespace gaps (gutters), and recursively split along the widest gap until each region contains only a single reading unit. For two-column pages, the vertical column gutter is usually the widest whitespace on the page, so the first cut separates the left and right columns; then within each column, horizontal gaps separate lines and paragraphs. XY-cut's advantages are determinism and no model training required; its weaknesses are sensitivity to borderless tables, rotated text, and complex magazine layouts, and the gap threshold needs tuning per document. The xy_cut() in the article's example is a minimal implementation of this idea, with a priority rule of "split columns first, then horizontal gaps."

Runnable Example: Two Columns + Table → Markdown → Chunking

Below we generate a small sample PDF with no external files needed, then compare the output of extract_text() with that of a ~90-line converter.

python -m venv .venv && source .venv/bin/activate
pip install pdfplumber==0.11.10 fpdf2==2.8.9

make_sample.py builds a page with a heading, two columns, and a bordered table:

**The difference between bordered and borderless tables** determines whether `find_tables()` works. pdfplumber's default strategy relies on straight lines (vector segments) on the page to infer cell boundaries, so it's reliable for tables with full borders. The "three-line tables" common in academic papers have only a top rule, bottom rule, and a header rule, lacking column separators—default parameters often fail to detect them. You need to use `vertical_strategy="text"` and `horizontal_strategy="text"` in `table_settings` so the algorithm infers column boundaries from text alignment. Even then, merged cells spanning rows and nested headers still cause errors, so in production you usually need manual spot checks or fall back to an ML table model (like Docling's TableFormer).
# make_sample.py - build a two-column PDF with headings and a bordered table
from fpdf import FPDF

LEFT = ("Retrieval quality depends on chunk boundaries. When one chunk mixes "
        "two sections, its embedding blurs both topics. Splitting at headings "
        "keeps each chunk about one thing.")
RIGHT = ("Tables are worse. Flattened into plain text, a row of values "
         "loses its column headers, so the model cannot tell which number "
         "belongs to which metric.")

pdf = FPDF(format="A4")
pdf.add_page()
pdf.set_font("Helvetica", "B", 20)
pdf.cell(0, 12, "Chunking PDFs for RAG", new_x="LMARGIN", new_y="NEXT")
y = pdf.get_y() + 4
for x, title, body in [(10, "Why boundaries matter", LEFT), (110, "Why tables break", RIGHT)]:
    pdf.set_xy(x, y)
    pdf.set_font("Helvetica", "B", 14)
    pdf.multi_cell(90, 8, title)
    pdf.set_x(x)
    pdf.set_font("Helvetica", "", 10)
    pdf.multi_cell(90, 5, body)
pdf.set_y(150)
pdf.set_font("Helvetica", "B", 14)
pdf.cell(0, 8, "Formats compared", new_x="LMARGIN", new_y="NEXT")
pdf.set_font("Helvetica", "", 10)
with pdf.table(col_widths=(60, 40, 40)) as table:  # fpdf2 draws borders
    for row in [("Output", "Headings", "Tables"), ("Plain text", "lost", "flattened"), ("Markdown", "# / ##", "pipe table")]:
        r = table.row()
        for cell in row:
            r.cell(cell)
pdf.output("sample.pdf")
print("wrote sample.pdf")

pdf_to_md.py does four things: group words into lines (splitting at column gutters), sort with recursive XY-cut (split columns first if a vertical gutter exists, otherwise split the widest horizontal gap), mark headings based on font size relative to the page median, and replace bordered tables with Markdown tables using pdfplumber's find_tables().

# pdf_to_md.py - minimal text-layer PDF -> Markdown -> heading-aware chunking
import statistics, sys
import pdfplumber

def lines_of(words, gap=12):
    """Group words into line fragments; wide horizontal gaps (column gutters) start a new fragment."""
    rows = {}
    for w in words:
        rows.setdefault(round(w["top"]), []).append(w)
    frags = []
    for ws in rows.values():
        ws.sort(key=lambda w: w["x0"])
        cur = [ws[0]]
        for w in ws[1:]:
            if w["x0"] - cur[-1]["x1"] > gap:
                frags.append(cur); cur = []
            cur.append(w)
        frags.append(cur)
    return [{"x0": f[0]["x0"], "x1": f[-1]["x1"], "top": min(w["top"] for w in f),
             "bottom": max(w["bottom"] for w in f), "size": f[0]["size"],
             "text": " ".join(w["text"] for w in f)} for f in frags]

def gaps(intervals):
    intervals = sorted(intervals); out = []; end = intervals[0][1]
    for a, b in intervals[1:]:
        if a > end:
            out.append((end, a))
        end = max(end, b)
    return out

def xy_cut(items, min_gutter=7):
    """Recursive XY-cut: split columns first if a gutter exists, otherwise split the widest horizontal gap."""
    if len(items) <= 1:
        return items
    width = max(i["x1"] for i in items) - min(i["x0"] for i in items)
    cols = [g for g in gaps([(i["x0"], i["x1"]) for i in items]) if g[1] - g[0] >= min_gutter]
    for a, b in sorted(cols, key=lambda g: g[0] - g[1]):          # widest gutter first
        left = [i for i in items if i["x1"] <= a]; right = [i for i in items if i["x0"] >= b]
        if min(max(i["x1"] for i in left) - min(i["x0"] for i in left),
               max(i["x1"] for i in right) - min(i["x0"] for i in right)) > 0.2 * width:
            return xy_cut(left) + xy_cut(right)
    rows = gaps([(i["top"], i["bottom"]) for i in items])
    if not rows:
        return sorted(items, key=lambda i: (i["top"], i["x0"]))
    a, b = max(rows, key=lambda g: g[1] - g[0])
    return xy_cut([i for i in items if i["bottom"] <= a]) + xy_cut([i for i in items if i["top"] >= b])

def page_to_markdown(page):
    tables = [t for t in page.find_tables() if len(t.extract()) >= 2]   # bordered tables only
    def in_table(w):
        return any(t.bbox[0] <= w["x0"] and w["x1"] <= t.bbox[2] and
                   t.bbox[1] <= w["top"] and w["bottom"] <= t.bbox[3] for t in tables)
    words = [w for w in page.extract_words(extra_attrs=["size"]) if not in_table(w)]
    body = statistics.median(w["size"] for w in words)
    items = lines_of(words)
    for t in tables:
        rows = [[(c or "").replace("\n", " ").strip() for c in r] for r in t.extract()]
        md = ["| " + " | ".join(rows[0]) + " |", "|" + "---|" * len(rows[0])]
        md += ["| " + " | ".join(r) + " |" for r in rows[1:]]
        x0, top, x1, bottom = t.bbox
        items.append({"x0": x0, "x1": x1, "top": top, "bottom": bottom, "size": body,
                      "text": "\n".join(md), "table": True})
    out = []
    for it in xy_cut(items):
        if it.get("table"):
            out.append(it["text"])
        elif it["size"] >= body * 1.8:
            out.append("# " + it["text"])
        elif it["size"] >= body * 1.3:
            out.append("## " + it["text"])
        elif out and not out[-1].startswith(("#", "|")):
            out[-1] += " " + it["text"]                  # merge line-wrapped paragraphs
        else:
            out.append(it["text"])
    return "\n\n".join(out)

def chunk_by_heading(md):
    chunks, current = [], []
    for block in md.split("\n\n"):
        if block.startswith("#") and current:
            chunks.append("\n\n".join(current)); current = []
        current.append(block)
    return chunks + ["\n\n".join(current)]

if __name__ == "__main__":
    with pdfplumber.open(sys.argv[1] if len(sys.argv) > 1 else "sample.pdf") as pdf:
        page = pdf.pages[0]
        print("=== naive extract_text() ===")
        print(page.extract_text()[:240], "...\n")
        md = page_to_markdown(page)
        print("=== markdown ===")
        print(md, "\n")
        for i, c in enumerate(chunk_by_heading(md)):
            print(f"--- chunk {i}: {c.splitlines()[0]!r} ({len(c)} chars)")

Run:

python make_sample.py && python pdf_to_md.py

In the extract_text() output, lines 3-5 all span both columns, with interleaved content:

Chunking PDFs for RAG
Why boundaries matter Why tables break
Retrieval quality depends on chunk boundaries. When Tables are worse. Flattened into plain text, a row of
one chunk mixes two sections, its embedding blurs both values loses its column headers, so the model cannot
topics. Splitting at headings keeps each chunk about tell which number belongs to which metric.
one thing.
Formats compared
Output Headings Tables
Plain text lost flattened
Markdown # / ## pipe table

Converter output (tested on Python 3.13, pdfplumber 0.11.10, fpdf2 2.8.9):

# Chunking PDFs for RAG

## Why boundaries matter

Retrieval quality depends on chunk boundaries. When one chunk mixes two sections, its embedding blurs both topics. Splitting at headings keeps each chunk about one thing.

## Why tables break

Tables are worse. Flattened into plain text, a row of values loses its column headers, so the model cannot tell which number belongs to which metric.

## Formats compared

| Output | Headings | Tables |
|---|---|---|
| Plain text | lost | flattened |
| Markdown | # / ## | pipe table |

--- chunk 0: '# Chunking PDFs for RAG' (23 chars)
--- chunk 1: '## Why boundaries matter' (196 chars)
--- chunk 2: '## Why tables break' (170 chars)
--- chunk 3: '## Formats compared' (134 chars)

Each column becomes an independent paragraph under its heading, the table keeps its header, and the chunker splits by heading. In a real pipeline, you'd merge tiny chunks (like chunk 0, which is just a heading) into the next one and add the heading path to the metadata.

Limitations of the Toy Version

This version only works on this one page; real documents will quickly expose problems:

• Paragraph detection: it concatenates all non-heading lines until the next heading. Real code needs to detect paragraph breaks (indentation, vertical spacing) and hyphenated words.
• Borderless academic tables: find_tables() with default settings won't detect "three-line tables."
• Headers and footers: rotated margin text (arXiv side stamps), headers, footers, and page numbers get mixed into the body.
• Symbol fonts: glyphs mapped to Unicode private use areas, with some fonts producing (cid:NN) garbage. Such pages should be flagged rather than embedded.
• Scanned pages: no text layer at all; they should be flagged rather than indexed as empty strings.

Key Takeaways

For native digital PDFs, most of the RAG quality lost at the extraction stage can be recovered by preserving reading order, headings, and tables. Start with a small rule-based converter like the one in this article, observe the chunking results, and only upgrade to a heavier ML pipeline when your documents truly require it. (This article was written with AI assistance; the code was actually run and the output is the real result.)

One-line Summary

Poor RAG performance often stems from PDF extraction losing structure; converting to Markdown and chunking by heading solves most of it.

Paragraph Summary

The article points out that poor RAG pipeline performance on PDF questions often doesn't stem from the embedding model or prompts, but from the extraction stage: line-by-line scanning breaks two-column reading order, tables get flattened and lose column headers, and headings become ordinary text lines, causing chunks to mix unrelated content and embeddings to blur. The author advocates converting PDFs to Markdown rather than plain text, because #/## headings serve as chunk boundaries and metadata, pipe tables preserve the correspondence between headers and values, and tools like LangChain's MarkdownHeaderTextSplitter can use this directly. The article compares the approaches, strengths, and licenses of libraries including pypdf, pdfminer.six, pdfplumber, PyMuPDF, Docling, Marker, MinerU, unstructured, and MarkItDown, and gives a selection rule: native PDFs with a text layer suit lightweight rule-based approaches, while only scanned documents need OCR/ML pipelines. Finally, it provides a ~90-line pdfplumber-based recursive XY-cut conversion script that was tested to correctly convert a two-column page with a table into Markdown and chunk by heading, while candidly acknowledging its limitations with paragraph detection, borderless tables, headers/footers, symbol fonts, and scanned pages.