Replacing Expensive LLM Calls in Financial Document Processing: A Practical Guide to SIE

In financial document processing pipelines, the most commonly overlooked cost black hole is often not analyst Q&A or management discussion summaries, but those high-frequency, low-variance "lightweight" LLM calls—such as asking "Is this an income statement?" once per PDF page. Each individual call looks cheap, but the sheer volume is enough to devour your budget. This article breaks down how to use task inventory, shadow runs, and confidence-based routing to migrate this work to specialized models on SIE (Superlinked Inference Engine), reserving frontier model pricing for the tasks that truly need frontier capabilities.

The Typical Shape of an Expensive Document Pipeline

Open a usage breakdown for a pipeline that processes PDF annual reports and client presentations, and the most glaring line is usually not a chat call. It's a page classifier that asks "Is this an income statement?" for every page of every PDF. Each call is cheap and nearly identical to the last, but those calls number in the millions. The model answering them has read most of the public internet yet is being paid to identify a table heading it could find with its eyes closed.

For U.S. EDGAR filings, such calls shouldn't exist at all. Every XBRL filing ships with a FilingSummary.xml and a set of rendered R files that name each statement individually, while the presentation linkbase arranges the order of line items within each statement. Statement mapping is free. The classifier is only valuable where no such metadata exists: foreign annual reports published as PDFs, investor presentations, data room uploads, and scanned attachments.

This is the typical shape of an expensive document pipeline: a general-purpose model doing high-frequency, low-variance work simply because it was the fastest thing to wire up during prototyping. The solution is to migrate this work task by task to specialized models on SIE, and to prove with evidence that the replacement holds up on your documents.

XBRL and EDGAR metadata: XBRL (eXtensible Business Reporting Language) is an XML-based financial reporting markup standard, and the U.S. SEC requires all EDGAR filings to be submitted in XBRL format. Every XBRL filing ships with a FilingSummary.xml that lists all rendered R files (such as R2.htm, R4.htm), each corresponding to a specific statement—income statement, balance sheet, cash flow statement, and so on. The presentation linkbase further defines the hierarchy and order of line items within a statement. This means that for EDGAR filings, page classification information is already fully provided by structured metadata, and there's no need to call an LLM to judge "Is this an income statement?" for every page. The classifier's true value is limited to scenarios where no such metadata exists: annual reports disclosed as PDFs by foreign issuers, investor presentations, data room uploads, and scanned attachments. Understanding this distinction is the first step to avoiding paying LLM fees for information that's already free.

Step 0: Take the Cheap Discounts First

Before migrating any task, check two things with your current provider.

Prompt caching. Classification and extraction prompts are mostly fixed text: instructions, label sets, output schemas, and a few examples, followed by a small chunk of page content. Put the fixed portion first and keep it byte-identical, and major hosted providers will bill the cached prefix at a discounted rate. On a prompt that's 90% boilerplate, note that discount rate, then inventory what's left that should still move to a specialized model.

Batch pricing. Most of this work isn't interactive. A quarter's worth of PDFs doesn't need an answer within a second. OpenAI's Batch API and similar offerings from other providers charge about half the synchronous price, at the cost of a completion window of up to one day.

Also check the obvious things: are you sending pages that don't need a model at all, such as EDGAR statements already named by R files, or cover pages and blank separators that a simple rule could discard? Measure the bill after these changes. Everything below is only worth it for what remains.

Step 1: Inventory Calls by Task, Volume, and Variance

You can't choose what to migrate without seeing the calls. Tag every LLM request with a task name, pull a month of logs, and build a table. The numbers below are fictional and are meant only to show the shape, not any real pipeline:

Task	Monthly calls (example)	Output shape	Input variance
PDF page classification (which statement is this?)	2,000,000	One label from a fixed label set	Low
Entity tagging (company, auditor, date)	600,000	Tagged spans	Low
Table field extraction (revenue, net income, EPS)	400,000	Fixed JSON schema	Medium
LLM relevance judgment for retrieved chunks	300,000	A score or yes/no	Low
Analyst Q&A across multiple filings	20,000	Free text	High
Management discussion summaries	15,000	Free text	High

Even if the numbers differ, the pattern usually holds: the highest-frequency tasks tend to have the smallest output spaces, and those are exactly the jobs that specialized models on SIE take on.

Record variance honestly. Low-variance tasks see inputs that resemble each other, with outputs drawn from a small, fixed space. High-variance tasks face new layouts, open-ended questions, or reasoning across multiple tables and filings.

Step 2: Pick Candidate Tasks with Fixed, Checkable Outputs

A task is a good candidate when it meets all three conditions:

• Fixed output: the answer comes from a closed label set or fixed schema
• High frequency: it runs many times per month on similar inputs
• Mechanically checkable: you can score the output against a label without a human reading prose

PDF page classification, entity tagging, relevance judgment, and table field extraction are all tasks that can migrate to specialized models on SIE.

Map each candidate task to the right specialized model type: a small generative model or trained classifier for page types, zero-shot entity models like GLiNER2 or NuNER_Zero for entity tagging, a cross-encoder reranker to replace an LLM being asked "Is this chunk relevant?", and a document model plus structured output model for table fields. The reranker replacement is usually the easiest win. A cross-encoder returns a calibrated score for each query-chunk pair, which is exactly what the LLM judge was being prompted to imitate.

Start with the highest-volume, simplest-output task. In PDF-heavy pipelines, that's usually page classification.

Cross-encoders and rerankers: A cross-encoder is a model architecture that concatenates a query and a document chunk and feeds them together into a Transformer, outputting a relevance score. Unlike a bi-encoder, which encodes queries and documents separately and scores them only via vector dot product, a cross-encoder lets every token of the query and document interact directly in the attention layers, giving higher scoring accuracy but also greater computational cost—each query-document pair requires a full forward pass. For this reason, cross-encoders are typically used as rerankers: the retrieval stage first uses a bi-encoder to quickly recall top-K candidates, then the cross-encoder re-ranks those K results. The "Is this chunk relevant?" task that the LLM judge is prompted to imitate is essentially the cross-encoder's scoring function. Replacing an LLM judge with a specialized reranker is usually the easiest migration win, because the two have identical output shapes and the reranker's scores are calibrated for direct use in confidence-based routing.

Step 3: Build an Evaluation Set First

Don't migrate a task without a gold set. For field extraction, the companion guide in this series, "Evaluating Extraction Accuracy", covers the methodology in detail: seed labels with XBRL facts where available, supplement with hand-labeled PDF pages, and score values, ratios, units, signs, periods, and entities separately. For classification and tagging, a few hundred labeled pages or passages per class is a reasonable starting point. For relevance judgment, take a sample of real queries and have humans score the retrieved chunks.

Include the pages that currently cause trouble: statements split across two pages, restated comparatives, note segments that look like primary statements. The replacement must hold up on these pages, not just on clean ones.

Step 4: Shadow-Run the Specialized Model

Run the specialized model on SIE in parallel with your current model, but don't use the specialized model's output. Log both answers for every request. After a week or two, you'll have three comparisons:

1. How each model performs against the gold set
2. How the specialized model compares to the current model on unlabeled live traffic
3. Latency and failure rates for both

Disagreements on live traffic are the most valuable output of a shadow run. Sample them, label them, and add them to the gold set. They tell you where to tighten the schema or switch model IDs on the same SIE client.

Step 5: Route by Confidence

Few switches need to be all-or-nothing. Most specialized models give you confidence signals: classifier scores, reranker scores, span confidence from entity models, or nulls returned by structured output models for missing fields. Use them:

• High confidence: adopt the specialized model's answer
• Low confidence or null: route to human review
• Log every review and its input so those cases become the next batch of gold set rows

Tune the threshold on the gold set to reach the required accuracy, and watch what share of traffic goes to review. If that share is high, narrow the task or try the next catalog model on the same SIE client.

Calibrated scores and threshold selection: Calibration means that a model's confidence score truly reflects its probability of being correct. For example, when a reranker gives a score of 0.9, good calibration means the judgment is about 90% likely to be correct. Specialized models (classifiers, rerankers, entity models) are usually easier to calibrate than an LLM's self-reported confidence, because their training objectives directly optimize the correspondence between scores and correctness, whereas an LLM's "confidence" often comes from token probabilities or verbal expression and is heavily influenced by the prompt. Calibrated scores make threshold selection in confidence-based routing well-founded: plot accuracy and fallback rate curves at different thresholds on the gold set, and choose the lowest threshold that meets the business accuracy requirement, thereby minimizing the share of traffic entering human review. If the fallback rate is still too high, the task definition or model choice needs adjustment, not simply a tighter threshold.

Step 6: Switch Over and Keep Monitoring

When the specialized model meets its accuracy threshold and the fallback rate is stable, make it the default path. Then keep watching:

☐ Weekly accuracy evaluation on a rotating sample of production traffic, with human labeling

☐ Fallback rate per task, with alerts on shifts

☐ Re-run the gold set after any model, prompt, or parser change

☐ A check after each reporting season, because new layouts appear in batches

Budget for migration costs too. If the migration also changes how you retrieve—for example, adding a new embedding model alongside a new reranker—that means re-embedding and rebuilding indexes for the entire corpus. Any new model means a new evaluation cycle. These are one-time costs, but they're real and should be planned for.

Where SIE Fits In

Once you've carved tasks off the frontier model, they need somewhere to run. Moving four tasks to four vendors trades one bill for four integrations. SIE from Superlinked unifies them behind one API: reranking, entity extraction, OCR, document-to-Markdown and structured output, and embeddings. You can use it as a managed service on SIE Cloud, self-host it in your own VPC, or run it locally. According to the Cloud pricing page (verified on 2026-09-30), GLiNER2 Large entity extraction is $0.186 per million input tokens, Qwen3-Reranker-0.6B is $6.63 per million pairs (top-50 reranking means 50 pairs, about $0.0003 per query), LightOnOCR-2-1B is $2 per 1,000 pages, and Qwen3.8-27B (FP8) backing document field extraction is $0.72 per million input and output tokens. Treat these as unit prices in your inventory.

For tasks already running through the OpenAI SDK, the Cloud page shows a migration pattern that only changes the base URL. For a priced Cloud shadow run, use Qwen3.8-27B (FP8) via native SDK chat calls. Once its labels agree with your current model, smaller catalog models like GLiNER2 classification tasks are a strong production path on the same API. A shadow page classifier looks like this:

from sie_sdk import SIEClient

client = SIEClient(api_key="sk-sie-…", base_url="https://api.superlinked.com")
shadow = client.chat_completions(
    "Qwen/Qwen3.8-27B-FP8",
    page_classification_messages,
    max_completion_tokens=16,
)
label = shadow["choices"][0]["message"]["content"]

Entity tagging uses the native SDK's extract call with a label list, and relevance judgment uses score, as shown in the SIE documentation. The engine is open source under Apache 2.0 (GitHub), so tasks you migrate to Cloud can later move to a self-hosted cluster with the same model IDs.

To be candid: SIE is where the tasks you migrate off general-purpose models live: OCR, extraction, embeddings, and reranking behind one API. On SIE Cloud, Superlinked runs the GPUs, and you pay per token, pair, or page, with GLiNER2, Qwen3 rerankers, LightOnOCR-2-1B, and Qwen3.8-27B (FP8) on the price list. Self-hosting the same Apache 2.0 engine on your own GPUs lets tasks you shadow-ran on Cloud move to your cluster with the same model IDs. The weights are open and fixed. Page classification, entity tagging, and relevance judgment share the same client as document conversion and field extraction, keeping four tasks with one vendor.

FAQ

Which call should I migrate first?

Migrate the one with the highest volume, the simplest output, and that isn't already answered by free metadata. For EDGAR filings, first check whether R files and the presentation linkbase have already made page classification unnecessary. For PDF annual reports and presentations, page classification usually comes first: the label set is small, volume is high, and errors visibly break downstream. Evaluate table field extraction on a field-level gold set, then run that extractor on the same SIE client.

Why can't I just turn on caching and batch and stop there?

Run Step 0 so your inventory reflects the discounted rates. Page classification, entity tagging, and relevance judgment are still high-frequency, fixed-output work, and SIE offers these specialized models behind one API, in the cloud or on your own GPUs. The per-call price on SIE Cloud is what you put next to your current bill.

Doesn't this just shift operational burden onto you?

On SIE Cloud, Superlinked runs the GPUs. Self-hosting the same Apache 2.0 engine on your own Kubernetes cluster, as described in the deployment guide, keeps documents and model IDs in your environment. Running locally is good for development. Either way, evaluation and routing stay in your code.

Pay Frontier Prices Only for Frontier Work

The most expensive part of most document pipelines is simple work done at scale. Use EDGAR metadata where it exists, inventory the rest, build a gold set, shadow-run, and route by confidence. SIE is where those specialized tasks run: one API, open weights, cloud or your own GPUs. SIE Cloud prices each per token, pair, or page so you can put those numbers next to your current bill.