Why Can't Multimodal LLMs Read Text That Humans Can Squint and See? SquintBench Exposes the Gap

When Machines Miss What Human Eyes Can Read

Humans have a remarkable ability: faced with a blurry, low-resolution image, we can often squint and make out the outlines of embedded text. Yet recent research shows that today's multimodal large language models (MLLMs) repeatedly fail in exactly this scenario — not only unable to read the text, but sometimes reporting that "there is no text in the image" at all.

Researchers have coined the term Squint Text for this phenomenon and frame it as a "recognition deficiency" rather than a comprehension problem. The motivation comes straight from production environments: in content moderation pipelines, large volumes of text embedded within the coarse structure of images are missed by models, directly undermining the reliability of moderation systems.

Two Benchmarks: From Real-World Scenarios to Controlled Diagnosis

To study this problem systematically, the team built two evaluation tools.

SquintBench: Real-World Challenges from Production

SquintBench contains 1,076 real images extracted from production moderation pipelines. These samples are not artificially constructed lab puzzles — they are actual cases where models failed during real deployment, making the benchmark both representative and compelling.

SquintCtrl: Dissecting Failure Mechanisms

Complementing it is SquintCtrl, a controlled synthetic image suite designed specifically to diagnose the mechanisms behind failures. By manipulating variables in a controlled setting, researchers were able to pinpoint exactly where in the pipeline information is lost.

Where Does It Go Wrong? Mechanistic Analysis with Frozen Weights

The most valuable part of the research is its deep dive into failure causes. By analyzing models with frozen weights, the team arrived at several key findings:

• Coarse signal present, glyph identity absent: Embedded text does leave a "coarse signal" inside the model, but the specific glyph identity representation is extremely weak. In other words, the model "vaguely senses something is there" but cannot identify what character it is.
• Spatial selection is the primary bottleneck: When researchers explicitly provided the correct spatial region, glyph identity became much easier to extract. This means the model lacks not recognition capability per se, but the ability to "place attention in the right location" — spatial selection is the critical chokepoint.
• High-frequency content interference: Under a fixed visual token budget, suppressing high-frequency content in the image actually improves reading performance. Additionally, the number of visual tokens itself may be a contributing factor.

These findings decompose a vague "can't read text" problem into specific, locatable, and actionable bottlenecks.

Glyph identity refers to a model's specific representation of "which character is this visual symbol" — distinct from merely perceiving that "there is some texture or edge here." Visual neural networks form hierarchical representations ranging from low-level edges to high-level semantics; glyph identity belongs to the high-level semantic layer and requires sufficient spatial resolution and attentional focus to be activated. The research found that low-resolution embedded text does trigger some low-level signal in the model's feature space, but this signal is too weak to drive high-level character classification decisions — which is precisely why models report "no text in the image": their confidence threshold is never crossed.

Visual token budget is another core constraint in how multimodal LLMs process images. To feed an image into the language model backbone, it must be encoded as a fixed number of tokens (e.g., 256 or 576), and this count directly caps the level of visual detail the model can "see." When an image contains abundant high-frequency texture (background noise, complex patterns), the limited token budget gets consumed by these distractors, further compressing and diluting low-contrast text regions — which also explains why suppressing high-frequency content improves text recognition.

Squint OPSD: Teaching the Student Model to "Read on Its Own"

Building on this diagnosis, the researchers propose Squint OPSD as a solution. Its core is a teacher-student distillation paradigm:

• Teacher with privileged context: A frozen-weight teacher model receives a downsampled view of the image along with the ground-truth text transcription as "privileged context."
• On-policy trajectory supervision: The teacher's token-level distribution supervises the student model's learning along on-policy trajectories.
• Independent inference: After training, the student model faces only the original image at inference time — no auxiliary views or extra inputs required.

The elegance of this design lies in using the "answer" and downsampled view during training to guide the student toward correct reading strategies, while at deployment the student operates entirely on its own — leveraging privileged information during training without adding any inference cost.

On-policy distillation differs from traditional knowledge distillation in where the supervision signal comes from. Traditional distillation typically has the student directly mimic the teacher's output distribution on fixed samples (off-policy), whereas on-policy distillation applies teacher supervision to the student's own generated trajectories — the teacher corrects the student "on the path it is currently taking" rather than requiring the student to replicate the teacher's path exactly. This approach better handles the gap between training and inference distributions (the exposure bias problem), and is especially stable for autoregressive generation tasks.

Privileged context originates from the Learning Using Privileged Information (LUPI) framework: during training, the model is given additional information that will not be available at inference time, to accelerate or improve learning — but the model must ultimately work independently without that information. In this study, the teacher simultaneously sees the downsampled view and the ground-truth transcription; the latter is the privileged context. It tells the teacher "what the correct answer is," enabling the teacher to provide more precise gradient signals to the student's token generation process, while the student at inference time faces only the original image with no external answer.

Results: ANLS Jumps from 26 to 88

The experimental results are striking. On SquintBench, Squint OPSD achieves an ANLS score of 88.61, far outpacing the baselines:

• Unadapted original backbone model: 26.50
• Best general-purpose model (original view): 24.44
• Best off-the-shelf model on downsampled view: 68.60

The leap from 26.50 to 88.61 means the gap between "visual evidence actually present in the image" and "the final readout generated by the model" has been dramatically narrowed — and all of this is achieved without any auxiliary views at inference time.

ANLS (Average Normalized Levenshtein Similarity) is the standard evaluation metric in document understanding and text recognition, measuring the similarity between model output and ground-truth answers. It is based on edit distance (Levenshtein distance) — the minimum number of insertions, deletions, and substitutions needed to transform the predicted string into the correct one — normalized to a range of 0 to 100 (or 0 to 1). Compared to exact match, ANLS is more forgiving of minor spelling errors, more fairly reflecting a model's actual reading ability: misreading "Hel1o" as "Hello" incurs only a small penalty rather than a zero score. An ANLS jump from 26.50 to 88.61 means character-level similarity between model output and ground truth rose from roughly one quarter to nearly nine-tenths — a breakthrough of significant magnitude for OCR-related tasks.

Implications for the Industry

The significance of this research goes beyond an impressive score. It exposes a systematically overlooked weakness in current multimodal models: in real-world scenarios such as content moderation, document understanding, and image forensics, low-resolution or embedded text is pervasive, yet mainstream models handle it far less capably than human intuition would.

More importantly, the research attributes the problem to actionable mechanisms — spatial selection and high-frequency interference — rather than simply blaming "the model isn't big enough." This provides a clear direction for future improvements: rather than blindly scaling parameters, the focus should be on targeted optimization of spatial attention allocation and visual token budget strategy.

For teams relying on multimodal models for content safety, this is also a wake-up call — the Squint Text being missed may be the most invisible vulnerability in the entire moderation system.