Verdandi: An Open-Source Document Intelligence Tool at One-Tenth the Cost of Azure

Document processing is a hidden pain point across many industries. A former EY (Ernst & Young) risk consulting analyst shared his self-built document intelligence platform, Verdandi, on Bilibili and GitHub — an open-source project that started from the simple desire to stop manually copying data from PDFs into Excel, and eventually grew into a challenger to Microsoft's Azure Content Understanding (ACU).

The question the project raises is straightforward: when document processing needs across global trade, healthcare, and insurance are essentially the same, why should every industry buy a closed, opaque, usage-billed tool? Verdandi's answer is: an open-source platform that is customizable, has replaceable models, and comes with a complete human review workflow.

Starting from the Daily Grind at EY

The author's work at EY can be summed up in one word — documents. According to his account, from analyst to senior consultant, team members spent more than 6 hours a day validating and extracting document data: opening PDFs, splitting screens with Excel, copying and pasting day after day. Regardless of title or experience, the task itself was no different.

What puzzled him even more was the incentive structure: billing by the hour seemed more directly rewarded than improving the process. This pushed a non-technical analyst to tackle "the most annoying technical problem." When he started building Verdandi in June 2026, he had no idea how complex the problem was — and that certain validation errors could directly affect financial figures and settlement timelines.

His initial goal was simply to find a tool that could convert non-OCR PDFs into OCR, but he found that document processing products on the open web were either scattered or extremely commercialized. Among the 7 to 9 global logistics document processing companies he researched, none offered a transparent product that end users could use directly — the options were always "book a demo" or "contact sales."

Azure Content Understanding initially gave him hope, but the experience of logging into Azure Foundry was described by him as "like stepping into the cockpit of an Airbus A350" — powerful but with an extremely high barrier to entry.

Azure Content Understanding (ACU) is a document understanding service under Microsoft's Azure AI Foundry ecosystem, an extension of the Azure AI Document Intelligence product line. It packages optical character recognition (OCR), layout analysis, table extraction, form field mapping, and LLM-based semantic understanding into a managed API. Users define a schema so the service can automatically extract structured data from invoices, contracts, customs declarations, and similar documents. ACU pricing is typically based on document pages or feature tiers (e.g., pure OCR, table recognition, form + table combinations) layered on top of each other, and it follows Azure region list pricing, so at scale the cost rises quickly with page count and feature complexity. For end users, the main barrier is that resource creation, model deployment, quota requests, and permission configuration must all be completed in the Azure portal — an enterprise-grade process that is quite unfriendly to individuals unfamiliar with cloud platforms.

Benchmark: 97% vs 96.2%, a 9.7x Cost Difference

Verdandi was benchmarked against Microsoft Azure Content Understanding on 100 documents. The test set included 35 FATURA2 clean invoices, 25 CORD v2 real Indonesian receipt photos, 10 Kaggle scanned invoices, and 30 synthetic documents (clean, degraded scans, handwritten style, Hindi + English mixed, multi-document).

Both systems processed the same 100 SHA256-verified documents (103 pages total), using the same 8-field schema plus line items, run zero-shot, and scored by the same grader with the same normalizer. Verdandi used GPT-6-luna, and ACU used its strongest supported models, GPT-5.5 and text-embedding-3.

Metric	Verdandi	ACU
Field macro accuracy (mean of two runs)	97.0%	96.2%
Line item precision/recall mean	95.4 / 96.3	94.2 / 95.3
Line item F1	0.958	0.947
Documents found	100/100	100/100

The accuracy gap is only 0.8 percentage points, and the author himself admits it's "just a few cells across two runs; ACU wins on several fields and messy photos." But the cost gap is far more significant:

Cost Item	Verdandi	ACU
Total cost (corrected)	$0.2619	$2.5522
Per document	$0.0026	$0.0255
Cost ratio	9.7x	—

Verdandi used more than twice the tokens (about 1.31 million vs 568,000), but its overall cost is far lower because its per-token rate through its subscription gateway is lower. ACU is billed at Azure East US list pricing, with different pricing tiers based on OCR, tables, and table + form features.

One detail: in the caveats, the author candidly notes that public datasets (FATURA2, CORD, Kaggle) may appear in either model's training data, and the 30 synthetic documents serve as an anti-contamination control group; Kaggle scans carry the highest risk, and both systems read incorrect values on kaggle_01. Verdandi intentionally adopts a day-first date strategy, so some US-style date formats were counted as misses on the Kaggle scans.

Human in the Loop: The VLM Is Not the Final Arbiter

One of Verdandi's core design principles is that "the VLM is not judge, jury, and executioner." The system does not give the vision model full decision-making authority.

Based on user-set confidence thresholds, the VLM flags uncertain extractions and validation points, and a fallback model examines the disagreement and provides a confident answer. If the fallback model also cannot reach a precise conclusion, the case is escalated as a "flag" for human intervention, visually confirming the actual value on the document.

Users can view where the "model claims" and the "text claims" the extracted data is located, and decide whether to agree, disagree, or enter a custom value.

The philosophical basis for this design is: a reliable system does not need to return an answer for every field — it should be able to say, "I don't have enough evidence to tell you confidently." Confidence is not the AI's level of certainty in making a claim, but the level of trust the system has earned for that claim.

Technical Architecture: Vision First, Verification Second

Document extraction is fundamentally spatial. An invoice number is not just a string — it exists at a specific location on a specific page, usually next to a label, inside a table, or within some visual structure.

Verdandi's OCR layer produces structured text information along with data coordinates, rather than simply dumping large blocks of text. This lays the foundation for the downstream text pipeline to independently verify the vision model's decisions.

The Text Oracle

When the vision model extracts INV-49281 and claims it was found in a certain region on page 1, the text oracle performs a less glamorous task: "Can I actually find INV-49281 in the text near that region?"

If found, there is independent evidence supporting the model's extraction. If not, Verdandi does not cleverly invent a substitute value — it raises a flag. For document processing systems, "close enough" can be wildly wrong.

The author specifically explains why he doesn't use another AI model for verification: if the first VLM says the invoice number is INV-49281, then asking another VLM "do you agree?" results in two probabilistic systems agreeing with each other, which does not constitute an independent source of truth.

Independent source of truth is a key principle in reliable system design: the verifier must reach its conclusion based on a different mechanism or data set than the thing being verified; otherwise, verification is just repeating the same error. In document processing, if another VLM is used to "review" the first VLM's output, both share similar training distributions and hallucination patterns, making it easy for them to make the same type of mistake on the same field — so it does not constitute effective verification. The text oracle takes another path: it does not re-understand the document's semantics, but instead returns to the structured text and coordinates produced by OCR and performs deterministic string and location matching. The trade-off is that it cannot judge whether the semantics are correct — only whether "the value the model claims actually appears at the location it claims." But precisely because of this, it provides evidence independent of the vision model, rather than yet another probabilistic opinion.

Confidence Is at the Field Level

A document should not receive a blanket "confidence of 87%." Imagine a customs document with 50 extracted fields, 49 of which are completely reliable and 1 of which is blurred due to scan damage — a document-level score would obscure this problem.

Verdandi evaluates confidence at the field level, allowing uncertainty to be attached to specific pieces of information, and enabling the system to distinguish between "high confidence in the invoice number" and "the tax amount needs review."

VLM (Vision-Language Model) is a class of models that accept both image and text inputs and output text. Representative examples include the GPT-4V/5V series, Claude's vision versions, and open-source multimodal models. Unlike LLMs that only process text, VLMs can directly "see" document screenshots or scans, and can therefore bypass traditional OCR pipelines to read field values, table structures, and layout relationships directly from pixels. In document intelligence, VLMs' advantage is handling handwriting, skew, and interleaved tables — scenarios where traditional OCR easily fails. Their disadvantage is that their output is inherently probabilistic generation, so they may "read it wrong but be very confident," and they struggle to provide a traceable chain of evidence. Verdandi is designed precisely to address this flaw with the text oracle and field-level confidence, leaving the VLM responsible only for candidate extraction, with independent text evidence and a human workflow to confirm.

Templates, Workflows, and Cross-Document Validation

Verdandi's template system gives users full control over how data is extracted. Users can describe a document (what it is, what it looks like, how to identify it among multiple documents) and add vocabulary unique to that document. The "Allow-no-match" feature lets the VLM choose to flag and give up when it is unsure whether a found document matches the description, rather than extracting data from an unrelated document.

Each data point has its own customizable confidence threshold. A higher threshold means the VLM's extracted data must be highly accurate, otherwise it is marked in the review area as "below confidence threshold."

For documents with line items such as invoices, users can extract line items and, in edge cases where header wording differs, enter data to help the VLM understand what is being extracted. Users can also set custom arithmetic rules, such as "sum of line items = invoice total," and table annotations explaining how to read the table.

Cross-Document Validation

This is a feature ACU does not have. When a supplier provides a file containing invoices and corresponding shipping documents, cross-document validation checks whether the documents in the file are related. Unrelated documents are flagged and can be manually reviewed in the "review area."

For example, the customs declaration number on an invoice is 1234567, while the number on the customs declaration is 1234568. Processed independently, both fields look plausible, both may have high confidence, and both correctly correspond to their respective documents — but one of them is wrong. Cross-document validation lets fields from different documents be compared against each other.

Verdandi does not automatically decide which document is wrong, but rather says: "These two items are inconsistent; someone should take a look." For financial and compliance documents, automatically "fixing" one value based on another can be worse than simply flagging the discrepancy.

Zero Data Retention (ZDR) is a data handling commitment offered by model providers: customer inputs and outputs are not used for training and are not stored long-term on the provider side — they exist only for the brief time necessary to complete inference. For sensitive documents in finance, healthcare, and legal fields, ZDR means that even if a document is sent to a cloud model, the provider cannot later retrieve it or use it to improve the model. Many enterprise APIs (such as certain enterprise offerings from OpenAI and Anthropic) offer ZDR options, but it usually requires a separate application or agreement. In hybrid architectures like Verdandi, ZDR works together with local OCR, the ability to exclude specific providers' models, and cleanup of disk images after export, forming a layered data governance strategy that lets users weigh convenience against privacy themselves.

Privacy and Data Security

• Local OCR: Verdandi can OCR documents entirely on-device, so documents never leave the machine.
• ZDR models: Through OpenCode, usage can be restricted to zero data retention (Zero Data Retention) models only, preventing providers from retaining submitted data.
• Provider control: Through OpenCode, Chinese models can be excluded from the available model pool.
• Disk cleanup: Users can enable "free disk after export" to delete locally generated document images after a successful export.

The goal is to keep sensitive processing as local as possible, minimize provider-side data retention, and let users control which models can see their documents.

Export and Observability

The final output layer is deliberately "boring." JSON is used for programmatic workflows, preserving the structured representation Verdandi generates; Excel is used for human follow-up processing, keeping line item tables as line item tables rather than turning them into ["Item 1", "Item 2", "Item 3", ...] inside cell A17.

Run Explorer exposes the metadata and intermediate information of processing runs: model usage, token consumption, extraction attempts, validation results, flags, processing events, errors, and failures. This lets users answer questions such as "why did this document cost more to process than the last one" or "which model consumed the most tokens in this workflow."

Honest Limitations

The author maintains a rare humility in the conclusion. Verdandi still has rough edges, and some aspects need further development. He plans to post the project on Reddit for "brutal feedback" — "if not Reddit, where else can you find a group of people who will be brutally honest about what you've built?"

Unimplemented features include self-learning (using human-validated outputs as context for future similar document processing) and a document interpreter (conversing about any detail of a document). He quotes a line from Arcane as a footnote on unfinished perfection: "Perfection has a cost. It's only a destination to strive for."

Verdandi is open-sourced on GitHub, built using models from Moonshot, DeepSeek^Ad, and OpenAI. For those tired of "book a demo" and hourly-billed document processing, this may be the first truly transparent, customizable, and cost-controllable option.