A New Approach to Document Grounding: PDF Geometry + OCR + Visual Evidence Achieves 72.6% Word F1

What Is Document Grounding?

In the field of intelligent document processing, extracting field values from a PDF is only the first step. The trickier challenge is: how do you precisely map those extracted values back to their exact locations in the original document? That's the core problem document grounding sets out to solve.

A Reddit developer shared their engineering work in this area and published a GitHub repository to invite community feedback. The most interesting constraint in their approach is that the entire grounding layer relies solely on information from the document itself — no external LLM inference or prior knowledge required. The solution currently achieves a 72.6179% Word Grounding F1 score on a benchmark.

Word Grounding F1 is the standard metric for measuring grounding accuracy — it's the harmonic mean of Precision and Recall. Precision measures what fraction of the system's predicted groundings are actually correct; Recall measures what fraction of all ground-truth locations were successfully found. F1 balances the two, with a perfect score of 100%. In the document grounding context, evaluation is done at the word level: the system's predicted bounding boxes must sufficiently overlap with the annotated ground-truth positions to count as a correct match. This metric is meaningful because it simultaneously penalizes both missed detections (low recall) and false positives (low precision), forcing the system to find a genuine engineering trade-off between coverage and accuracy. A 72.6% Word F1 means roughly one in four cases still has positional errors at the word level — a number that reflects the inherent difficulty of the task while also providing a clear quantitative target for future improvement.

Four Complementary Signal Sources

The technical approach combines multiple signals derived entirely from within the document, forming a set of mutually reinforcing grounding cues:

PDF Character Coordinates

For native (text-selectable) PDFs, individual characters carry precise coordinate information. This is the most reliable grounding source — each character's position on the page is definite and can be used directly to trace the physical location of extracted values.

OCR Spatial Geometry

When a PDF is a scan or an image-based document, character coordinates are unavailable. In this case, the system falls back on OCR recognition results and their associated spatial geometry (bounding boxes, line and column relationships, etc.) to supplement grounding. OCR geometry is the primary mechanism for grounding scanned documents.

Modern OCR engines — such as Tesseract, Google Document AI, and AWS Textract — not only return recognized text strings but also output bounding boxes for each result: rectangular regions in the page coordinate system (expressed in pixels or normalized coordinates), at the word, line, or even character level, along with confidence scores. In document grounding, this spatial geometry forms the core grounding evidence for scanned or image-type PDFs. However, OCR geometry has inherent error: scan skew, insufficient resolution, or complex backgrounds can all cause bounding box offsets or inaccurate word segmentation. This is precisely why the approach also incorporates string matching and visual evidence as cross-validation.

String and Format Matching

Extracted values must be matched against the text in the document. This includes both exact string matching and format-level matching (e.g., for dates, monetary amounts, identifiers, and other structured formats), enabling alignment even when minor recognition errors are present.

Table Cell Assignment and Page-Level Visual Evidence

Table scenarios require dedicated cell-assignment logic to place values in the correct row and column. Additionally, page-level visual evidence acts as a fallback signal, providing extra clues when the other sources aren't sufficiently conclusive.

Failure Cases Are Where the Real Challenges Live

The author frankly admits that what interests them most right now isn't the parts that already work — it's the remaining ~27% of failure cases. These are broken down into several categories:

• Direct grounding errors: Relatively straightforward technical defects that can be progressively addressed through algorithmic improvements.
• Annotation convention disagreements: Some "errors" actually stem from differences in annotation standards. The model's predicted location disagrees with the annotated expectation, but isn't necessarily wrong.
• Dense layouts: When page layouts are extremely compact with elements tightly packed together, the discriminative power of geometric signals drops significantly.
• Insufficient geometric distinctiveness: Multiple candidate locations are geometrically very similar, making it impossible to determine the correct target from coordinates and bounding boxes alone.

This careful breakdown of failure modes has more engineering value than simply chasing a higher score. It reveals the fundamental difficulty of document grounding: when pure geometric signals fail, the system needs to incorporate higher-level semantic or contextual understanding.

"Annotation Conventions" refers to how human annotators define the "correct answer" when building benchmark datasets. For example, when the amount field "$1,200.00" appears in a table cell, an annotator might only have drawn a box around the numeric portion, while the system located the full token including the currency symbol. Or date annotations might be at the line level rather than the word level. These disagreements aren't algorithmic failures — they reflect ambiguity in the evaluation standard itself. Across document intelligence datasets (such as FUNSD, CORD, and DocVQA), definitions of annotation granularity and boundaries are far from uniform. A prediction that looks like an "error" on one dataset could be perfectly correct under a different annotation scheme. Identifying and isolating these "false failures caused by annotation noise" is critical for accurately evaluating a system's true capabilities.

Implications for Document Vision and Grounding Research

72.6% Word F1 isn't a headline-grabbing number, but the value of this approach lies in its methodological path — grounding entirely based on observable signals from within the document (coordinates, OCR geometry, format, visual evidence) rather than relying on black-box inference. This makes the results interpretable and, when failures occur, makes it much easier to trace them back to a specific signal layer.

For developers working in document vision, information extraction, and grounding, this work offers several transferable insights: multi-signal fusion is more robust than any single source; tables and dense layouts require dedicated handling; and the classification of failures behind an evaluation score is often more actionable for guiding the next round of optimization than the score itself.

The author has open-sourced the repository and is actively seeking community feedback — a reflection of the fact that document intelligence remains in a phase of rapid iteration that depends on open collaboration. For teams looking to deploy document extraction and grounding capabilities in production, following open-source work like this is an effective way to stay at the technical frontier.

Summary

This work integrates PDF geometry, OCR spatial information, string/format matching, table cell assignment, and visual evidence into a document grounding layer that requires no external prior knowledge, achieving an interim result of 72.6% Word F1. Even more instructive is its taxonomy of remaining failure cases — annotation conventions, dense layouts, and insufficient geometric distinctiveness — which point clearly toward where future work on document grounding needs to focus.