Gemini 3.1 Flash-Lite
A lightweight language model in Google's Gemini series, designed with low latency and low cost as primary objectives, and suited for high-frequency inference scenarios.
Core Facts
Timeline (last 90 days)
The test covered four vision models: Claude Sonnet 5 (Anthropic), Gemini 3 Flash (Google), Gemini 3.1 Flash-Lite (Google), and GPT-5.4 nano (OpenAI).
Fabrication rates when the total is fully obscured: Claude Sonnet 5 at 38%, Gemini 3 Flash at 21%, Gemini 3.1 Flash-Lite at 44%, GPT-5.4 nano at 73%; fabrication rates when the total is half-obscured: Claude at 92%, the other three models all at 100%.
The median confidence difference across models when fabricating totals is enormous: Gemini 3 Flash and Flash-Lite at 95, GPT-5.4 nano at 62, and Claude Sonnet 5 at only 20.
When asked directly whether the total was visible, Claude, Gemini 3 Flash, and Flash-Lite answered across all 48 receipts that the total was not visible and could not be calculated, yet when asked directly for the total, they still fabricated one 21–44% of the time.
When the total was obscured but each line item was visible, Claude and Gemini 3 Flash got all 48 correct, with all failures coming from the smaller models (Flash-Lite failed 31%, nano failed 79%), and nano got only 12/48 correct even when explicitly instructed to add up the items.
After rebuilding the configuration under the same public identifier on the same day, the model-minus-uniform endpoint value decreased by 0.0429 (H-minus-R comparison 95% CI: [+0.0182, +0.0667]), and with six configuration components changing simultaneously it was impossible to isolate which component caused the difference.
All Facts (6)
The test covered four vision models: Claude Sonnet 5 (Anthropic), Gemini 3 Flash (Google), Gemini 3.1 Flash-Lite (Google), and GPT-5.4 nano (OpenAI).
65%UnverifiedFabrication rates when the total is fully obscured: Claude Sonnet 5 at 38%, Gemini 3 Flash at 21%, Gemini 3.1 Flash-Lite at 44%, GPT-5.4 nano at 73%; fabrication rates when the total is half-obscured: Claude at 92%, the other three models all at 100%.
50%UnverifiedThe median confidence difference across models when fabricating totals is enormous: Gemini 3 Flash and Flash-Lite at 95, GPT-5.4 nano at 62, and Claude Sonnet 5 at only 20.
50%UnverifiedWhen asked directly whether the total was visible, Claude, Gemini 3 Flash, and Flash-Lite answered across all 48 receipts that the total was not visible and could not be calculated, yet when asked directly for the total, they still fabricated one 21–44% of the time.
50%UnverifiedWhen the total was obscured but each line item was visible, Claude and Gemini 3 Flash got all 48 correct, with all failures coming from the smaller models (Flash-Lite failed 31%, nano failed 79%), and nano got only 12/48 correct even when explicitly instructed to add up the items.
50%UnverifiedAfter rebuilding the configuration under the same public identifier on the same day, the model-minus-uniform endpoint value decreased by 0.0429 (H-minus-R comparison 95% CI: [+0.0182, +0.0667]), and with six configuration components changing simultaneously it was impossible to isolate which component caused the difference.
50%