The Reading Order Challenge in PDF Parsing: How the 1.3B Model OXR Breaks Through

When processing PDFs of two-column papers, text extractors often read across both columns, causing each sentence to be stitched together from fragments of two different sentences. Tables are flattened into a jumble of words, formulas become scattered symbols, and headers and footers get mixed into body paragraphs. Once these structural errors are introduced, downstream NER or summarization models can hardly fix them. The SoMarkAI team has recently open-sourced OXR, a 1.3B-parameter document parsing model, attempting to solve the reading order problem at its root.

Reading Order: The Hidden Saboteur in PDF Parsing

PDF was originally designed for printing and does not preserve semantic structure. Extraction tools must infer on their own which text belongs to the same paragraph, how tables are organized, and how formulas are distinguished from body text. Two-column layouts are a classic trap: naive extractors scan from left to right by coordinates, concatenating the end of a line in the left column with the start of a line in the right column into one sentence. Downstream NLP tasks—entity recognition, relation extraction, summarization—depend on coherent sentence boundaries. Once sentences are cut in half, model performance silently degrades.

According to the original Reddit post, this kind of error is something "no downstream step can fix." The team spent a lot of time dealing with this problem and ultimately decided to build a dedicated model.

The underlying structure of a PDF is essentially a set of drawing operators. The positions of text blocks are determined by coordinate transformation matrices and contain no semantic information such as paragraphs, headings, or tables. Therefore, extraction tools must infer reading order through heuristic rules or machine learning models. Common heuristic methods include recursive segmentation based on XY-cut and projection analysis based on blank regions, but these easily fail in complex layouts (such as two-column layouts, nested tables, and mixed text-image arrangements). Reading order errors not only affect readability but also break sentence boundaries, coreference relations, and logical structure, leading to degraded input quality for downstream tasks such as NER, relation extraction, and summarization. Unlike OCR errors, reading order errors usually manifest as disordered text fragment order or cross-column concatenation, which are difficult to automatically correct with language models. Therefore, accurate modeling is required at the parsing stage.

OXR's Design Approach: Decoupling Layout Analysis and Joint Prediction

OXR accepts PDFs or page images and outputs Markdown or JSON. Each element carries a type label (13 types in total, covering tables, formulas, code, etc.), bounding box coordinates, and reading order. Headers and footers are discarded by default but can be retained.

Two design choices contribute most of the effectiveness. First, layout analysis is split into an independent model that users can replace. Pipelines that need different region labels can bring their own layout model without modifying the recognition and ordering modules. Second, recognition and reading order are jointly predicted in a single forward pass of the same model. This joint modeling allows the model to consider layout positions while recognizing text, avoiding the accumulation of errors from the two-stage approach of recognizing first and ordering later.

Layout analysis is a key subtask in document parsing, aimed at detecting regions such as text blocks, tables, images, and formulas on a page and assigning semantic labels. Traditional pipelines usually separate layout analysis from text recognition: first detect regions, then perform OCR or formula recognition on each region, and finally sort them according to region positions. This staged approach easily leads to error accumulation, and layout models are difficult to adapt to different domains. OXR splits layout analysis into an independent model, allowing users to replace it with custom models, thereby improving flexibility. Joint prediction of recognition and reading order draws on the idea of multi-task learning, letting the model learn positional relationships while recognizing text and reducing inconsistency between the two stages. Outputting all elements and their order in a single forward pass also reduces inference latency.

Training Strategy: Error-Sample-Driven Iteration

The post-training stage adopts a targeted data augmentation loop. Samples on which the model makes mistakes or shows uncertainty during training are converted into textual rationales. After clustering the rationales, each cluster drives the generation of targeted training data for the next round. This method focuses training data on the model's weak points rather than covering them uniformly.

On public benchmarks, OXR ranks first on olmOCR-Bench, OmniDocBench, and DocParsingBench. The team also acknowledges that the model still has weaknesses, one of which is multi-column layouts. Local execution requires a 24 GB NVIDIA GPU, which poses a barrier for individual developers. The team provides a free hosted API for testing your own files, but the shared queue is not suitable for production environments.

Error-driven iterative training is an active learning strategy whose core is to let the model identify its own weak points and generate targeted training data. Specifically, samples on which the model predicts incorrectly or with low confidence on the training or validation set are filtered out, converted into textual descriptions through rules or manual annotation (such as "this table spans columns, causing a reading order error"), and then clustered to form several error-type clusters. Each cluster represents a typical error pattern, and training samples targeting that pattern are then generated using data synthesis or augmentation techniques and added to the next round of training. This method avoids the data redundancy caused by uniform sampling and enables the model to converge faster to key capabilities with limited data. Similar ideas have been applied in OCR and document understanding, but OXR systematizes them for the joint task of reading order and element recognition.

Significance and Limitations for NLP Workflows

For NLP practitioners working with scanned or native digital PDFs, OXR's value lies in intercepting structural errors upstream. Element types, bounding boxes, and reading order are output together, so downstream tasks can directly consume structured JSON without having to guess paragraph boundaries from raw text.

The limitations are also obvious: multi-column layouts remain a weakness, and multi-column layouts happen to be the most common format for academic papers. The 24 GB VRAM requirement excludes users of consumer-grade GPUs. The shared queue of the free API means latency is uncontrollable. The team provides code and weights in the GitHub repository SoMarkAI/OXR, and the benchmark report is located at docs/benchmark.md.

The original post ends by asking NLP practitioners: which structural errors cost you the most? This may hint that the team's next phase will prioritize the error types most strongly reported by the community.

Downstream NLP tasks are highly sensitive to the structural integrity of input text. For example, named entity recognition (NER) depends on sentence boundaries and context windows; if sentences are truncated or concatenated, entities may be split or incorrectly merged. Relation extraction requires syntactic paths between entities, and reading order errors disrupt dependency structures. Summarization models depend on the coherence of paragraphs and sections, and cross-column concatenation leads to semantic drift. Therefore, structural errors at the document parsing stage often cannot be compensated for by fine-tuning downstream models; reading order correctness must be ensured at the source. OXR outputs structured JSON with type labels, bounding boxes, and reading order, allowing downstream tasks to directly consume elements such as paragraphs, tables, and formulas and reducing heuristic segmentation of raw text. However, multi-column layouts remain a weakness, and common documents such as academic papers and technical reports are precisely dominated by two-column formats, meaning OXR's coverage in practical applications is limited. In addition, the 24 GB VRAM requirement excludes most consumer-grade GPUs, and the shared queue of the free API is not suitable for production environments. The team needs to further optimize model efficiency or provide a lightweight version.

One-line Summary