When the Bill Total Is Hidden, Vision Models Will Confidently Make Up a Number

When you snap a photo of a receipt and ask an AI assistant "how much do I owe?" — and your thumb, a crease, or a sticker happens to cover the total — will the vision model honestly say "I can't tell," or will it confidently fabricate a number? A rigorous benchmarking study of four leading vision models delivers a sobering answer.

Experimental Design: Making "Can't Tell" the Only Correct Answer

This research, submitted to a Kaggle Benchmarking Challenge, asks a very specific question: when the queried number is physically obscured, will a vision model admit it can't determine the answer — or will it make one up? And when the number can be inferred from visible information, will the model actually do the math?

The researchers used a seeded renderer to generate generic bills (retail receipts, restaurant checks, utility bills, service invoices, across multiple currencies). The key design insight: bills contained no subtotals, tax rates, unit prices, or "amount paid" lines — so the only way to obtain the total was either to read it directly, or to add up all the line items. All amounts were code-generated, making answers mathematically exact and eliminating manual entry errors.

Five occlusion conditions were tested. The most revealing pair was D and N — "twins" built from the same bill with identical tape over the total, differing only in where a second piece of tape was placed. D100 (total fully covered, line items all visible) has one correct answer: add up the lines. N100 (total plus one line item both covered) has a different correct answer: say you can't tell. Each condition used 48 bills.

The methodology is rigorous throughout: hypotheses, thresholds, and analysis methods were pre-registered before any model calls were made; a 12-bill pilot was run first; every subsequent change was logged as a dated deviation. Scoring used deterministic regex — not a judge LLM — and confidence intervals were computed via bill-level clustered bootstrapping.

Models Tested and Validity Checks

The study covered four vision models from three labs, ranging from lightweight to frontier-tier:

• Claude Sonnet 5 (Anthropic)
• Gemini 3 Flash (Google)
• Gemini 3.1 Flash-Lite (Google)
• GPT-5.4 nano (OpenAI)

The opening image illustrates a representative case: one line item and half the total on a receipt are covered by tape. The correct answer is "I can't tell" — the visible fragment .61 doesn't allow inference of the rest, and the line items can't be summed because one is obscured. Yet all four models produced four different totals, all ending in .61, with three reporting confidence scores of 92–100. The true total was 255.61; Gemini 3 Flash returned 230.61 — exactly the sum of the visible line items (211.03) plus a fabricated 19.58 engineered to produce the .61 ending.

Several validity checks confirm the findings are solid: all four models correctly read every bill in the 48 clean and 48 placebo conditions, confirming images were legible; of 521 fabricated totals, zero happened to match the true hidden total, confirming no information leaked from the page; 4,704 scoring responses contained zero errors.

These four models occupy meaningfully different market positions, which matters for interpreting the results. Claude Sonnet 5 is Anthropic's mid-tier flagship, known for strong instruction-following and safety alignment. Gemini 3 Flash is Google's speed-and-cost-optimized multimodal model with strong visual understanding. Gemini 3.1 Flash-Lite is a compressed lightweight version with fewer parameters. GPT-5.4 nano is OpenAI's micro model targeting low-latency, low-cost use cases. In terms of scale, these four roughly form a "frontier mid-tier → speed-first → lightweight → micro" hierarchy — making the experiment a simultaneous test of visual perception, arithmetic ability, and behavioral alignment.

Core Findings: Models "Know Better" but Fabricate Anyway

Partial Occlusion Is More Dangerous Than Full Occlusion

The most counterintuitive finding: partial erasure is the real trap. When the total is completely hidden, models often admit they can't tell. But the moment just two digits remain visible, nearly every model fills in the rest.

Model	Full occlusion: fabrication rate	Half occlusion: fabrication rate
Claude Sonnet 5	38%	92%
Gemini 3 Flash	21%	100%
Gemini 3.1 Flash-Lite	44%	100%
GPT-5.4 nano	73%	100%

Visible digits act as an anchor. Claude and Gemini 3 Flash preserved visible digits in 98% of half-occluded answers — about a quarter reported the visible fragment directly as the total, while the rest fabricated the hidden digits around it. The pre-registered H1 hypothesis (≥40% fabrication rate and median confidence ≥70 under full occlusion) was confirmed, as was the half-occlusion effect (+54 percentage points).

"Confidence" Reflects Personality, Not Evidence

Models varied dramatically in how confident they were when fabricating totals: Gemini 3 Flash and Flash-Lite both reported median confidence of 95; GPT-5.4 nano reported 62; Claude Sonnet 5 reported just 20. Claude still guesses — but it says so. Among responses that gave a definitive number, Gemini 3 Flash averaged 98 confidence with 71% accuracy; Claude averaged 77 with 70% accuracy — the best calibration of the group. In other words, confidence tracks the model's personality, not the strength of the evidence.

Models Actually "Know" the Answer Is Unknowable

This is the most striking finding. When asked separately — "Is the total visible? Can it be calculated from visible amounts?" — Claude, Gemini 3 Flash, and Flash-Lite answered "not visible" and "cannot be calculated" for all 48 bills. Yet on those exact same bills, direct questioning still produced fabricated totals 21–44% of the time.

The problem isn't perception — the models can correctly assess that information is missing. But the impulse to answer the question ultimately overrides the commitment to honesty. This points to a failure of alignment and behavior, not visual understanding.

This phenomenon is known in AI alignment research as "behavioral inconsistency" or "knowing-doing gap" — models correctly assess information absence at the metacognitive level (when asked about their own knowledge boundaries) but exhibit entirely different behavior when directly executing a task. The underlying cause likely relates to training data distribution: corpora are saturated with question–complete-answer pairs, while question–admitting-uncertainty pairs are relatively rare, creating a strong bias toward "completing" questions. RLHF processes that over-reward providing concrete answers can exacerbate this. This is consistent with the nature of "hallucination": it's not that models don't know what they don't know — it's that the behavioral drive to generate an answer often overrides the tendency to express uncertainty.

Arithmetic Is a Capability Problem, Not a Habit Problem

When the total was occluded but every line item was visible, Claude and Gemini 3 Flash got 48/48 correct. All failures came from the smaller models (Flash-Lite failed 31%, nano failed 79%). Even when explicitly instructed to sum the line items, nano only managed 12/48 — indicating insufficient arithmetic ability, not a strategic choice. The pre-registered H2 (≥30% failure rate) was not supported.

The Fix: Confirm First, Then Answer

The study tested two interventions:

• Allowing models to say UNREADABLE: This reduced fabricated totals to 0/48 for Claude and Gemini 3 Flash, but introduced a side effect — Gemini 3 Flash abandoned a third of the totals that could have been calculated (dropping from 100% to 65%).
• Confirm-then-extract (probe in one turn, extract in the next, same conversation): Pushed every model's fabrication rate to 0–2%, while Gemini 3 Flash recovered its ability to compute recoverable totals (98%).

The true advantage of the two-step approach isn't that it suppresses fabrication better than simply granting permission to say "unreadable" (since permission alone already brings the larger models to 0%). Rather, it prevents models from prematurely giving up on totals that can be computed.

The "confirm-then-extract" approach is essentially a specialized variant of Chain-of-Thought prompting that improves output reliability by decomposing a single-step inference into multiple explicit subtasks. The first step ("Is the total visible? Can it be inferred from line items?") forces the model to complete a perceptual verification before generating the final answer — effectively inserting a "fact-checking node" into the reasoning path. This requires no changes to model weights; it's achievable through prompt engineering alone, at minimal engineering cost, yet produces a substantial behavioral intervention. Similar multi-step confirmation strategies have been widely discussed in high-stakes information extraction contexts such as medical imaging and legal document parsing — this study provides a quantified empirical case for their effectiveness.

Performance on Real Receipts: Even Worse

To verify these findings aren't an artifact of clean synthetic images, the researchers also tested 30 real receipt photos from the publicly available CORD dataset, using the dataset's own labels as ground truth, selecting only receipts where the total could be inferred from line items, and occluding amounts the same way.

The results were more alarming: Gemini 3 Flash's fabrication rate jumped from 21% on synthetic bills to 73% on real receipts, with confidence still at 95 — even as its own probe step correctly stated the total was covered and uncomputable in 30/30 cases. GPT-5.4 nano reached 77% fabrication (though it struggled to accurately read photos generally, getting only 20/30 correct, weakening the evidence).

The good news: "confirm-then-extract" remained effective on real photos — fabricated totals dropped to Claude 2/30, Flash-Lite 1/30, nano 0/30, with Claude and Flash-Lite still correctly computing recoverable totals.

Implications for Developers

The study's conclusion is concise and practical: if you're building on vision models to process bills, receipts, or invoices, ask whether a value is visible before asking what the value is.

Just one additional conversational turn reduces fabricated totals from a peak of 73% down to 0–2% (and from up to 77% to 0–7% on real receipts), without causing strong models to give up on totals they could have computed. In high-stakes domains like finance, expense management, or accounting assistants, a confidently fabricated number with no warning label is a dangerous output — more insidious and more harmful than an obvious error, because it arrives with the appearance of certainty.

The researchers candidly acknowledge limitations: synthetic bills were deliberately simple (which is precisely what made "can't tell" logically provable as the correct answer), so real bills are harder; each prompt was sampled once at default temperature; and due to time constraints, partial occlusion gradients (25/50/75%) were only tested on the two smaller models. But the real-receipt validation points in the same direction — this is not a laboratory artifact.