SLVR: Enhancing Multimodal Visual Understanding with Structured Latent Reasoning

Multimodal large language models (MLLMs) often rely on language priors rather than genuinely relevant visual evidence when answering visual reasoning questions — in other words, they tend to "guess" rather than truly "look." A new study on arXiv introduces Structured Latent Visual Reasoning (SLVR), which aims to bridge the gap between explicit chain-of-thought reasoning and latent reasoning, offering a new training paradigm for fine-grained visual reasoning.

The Root Problem: Why Models Don't Actually "Look" at Images

A persistent issue with current multimodal models is their tendency, when faced with visual question answering tasks, to draw on commonsense priors already embedded in the language model rather than attending to the actual evidence in the image. For example, when asked "Is the cup to the left or right of the table in this image?", the model may answer based on statistical patterns in its training data rather than the actual scene.

Text-based Chain-of-Thought (CoT) reasoning partially addresses this problem by encouraging the model to decompose visual questions into a series of intermediate evidence-seeking steps — progressively localizing and reasoning through the image. However, this approach comes at a clear cost: intermediate steps must be generated autoregressively token by token, which raises inference overhead considerably.

Latent Reasoning takes a different route — rather than generating explicit textual rationales, it completes reasoning within a hidden space, thereby avoiding decoding costs. But its drawbacks are equally significant: existing methods offer little control over what the intermediate states actually encode, making it difficult to apply targeted supervision to different reasoning stages such as planning, grounding, or evidence selection.

The language prior problem has structural roots in multimodal model architectures. Mainstream MLLMs typically convert visual encoder outputs into token sequences that are then processed by a language model backbone. Since the language model has been pretrained on vast amounts of text, its parameter space already stores a wealth of statistical regularities about the world. When visual tokens compete with language priors, the model tends to "favor" text-side evidence at the attention mechanism level. This phenomenon is especially pronounced on benchmarks such as MMStar and MMVP, which are specifically designed to detect the degree of visual dependency — these benchmarks deliberately include questions that cannot be answered correctly through language priors alone, allowing researchers to quantify the ratio of model "guessing" versus "observing." Latent Reasoning, as an emerging research direction, centers on performing the entire reasoning process within a continuous hidden vector space, bypassing autoregressive token-by-token text generation and compressing inference latency to near the level of a single forward pass. The tradeoff is that intermediate states are completely opaque to humans and difficult to supervise precisely using annotated data.

SLVR's Core Idea: Giving Latent Reasoning a "Functional Skeleton"

SLVR's key innovation lies in organizing multimodal reasoning into several typed latent stages, each corresponding to one of four steps in human-like reasoning: planning, grounding, evidence selection, and reasoning integration.

This design allows latent reasoning to retain the efficiency advantage of not generating an explicit text chain, while also gaining the clear functional structure characteristic of explicit chain-of-thought. Rather than an opaque flow of hidden states, each stage has a well-defined role and a corresponding supervision signal.

Two-Phase Training Strategy

SLVR's training proceeds in two steps.

The first step is forcing the model to rely on the image. Researchers mask portions of the text that would "leak" the answer, and use visually plausible distractors alongside correct answers to compel the model to extract information from the image rather than textual shortcuts. This fundamentally corrects the model's tendency to ignore the visual content.

The second step is structured, stage-by-stage supervision. Reasoning is organized into the four latent stages — planning, grounding, evidence selection, and integration — each supervised by its corresponding signal: planning by plans, grounding by bounding boxes, evidence selection by visual evidence, and integration by the final reasoning rationale. This per-stage supervision endows latent-space reasoning with interpretable functional specialization.

The "typed latent stages" design is typically implemented using special stage-separator tokens or learnable stage embeddings, enabling recognizable structural transitions in the model's hidden states across different reasoning steps. This differs from works like Coconut (continuous chain-of-thought), which allows the model to iterate freely in hidden space without imposing stage divisions — SLVR explicitly requires each latent state to align with a specific semantic function (planning / grounding / evidence / integration). The grounding stage's correspondence to bounding box supervision is particularly critical: bounding box coordinates are spatially discrete and precise signals that force hidden states to encode concrete spatial location information rather than remaining at an abstract semantic level. This "functional skeleton" approach also renders SLVR's reasoning process partially auditable — even though intermediate steps are not output as natural language, researchers can still decode the hidden states at each stage to verify whether the model truly performed grounding or evidence selection.

Experimental Results: Consistent Gains Across Multiple Benchmarks

Built on top of Qwen2.5-VL-7B, SLVR achieves consistent improvements across multiple multimodal reasoning benchmarks. The most striking results include:

• MMVP: absolute improvement of +9.4
• BLINK Relation: absolute improvement of +14.2

Additional improvements were also observed on V*, MathVista, ChartQA, and other benchmarks. These results span a variety of task types — spatial relations, mathematical visual reasoning, chart comprehension, and more — indicating that the benefits of structured latent supervision are not confined to any single scenario.

The researchers argue that these experiments validate the core hypothesis: structured latent supervision can enhance fine-grained visual reasoning while avoiding the decoding overhead of text-based chain-of-thought. In other words, it is possible to have it both ways — reasoning quality and reasoning efficiency need not be mutually exclusive.

MMVP (Multimodal Visual Perception) is a benchmark specifically designed to measure a model's visual perception ability. Its core design principle is to exclude any question that can be answered correctly without looking at the image, ensuring that every question's answer must be read directly from the visual content. BLINK is another benchmark emphasizing visual perception and spatial reasoning; its Relation subset focuses specifically on testing spatial relationships such as relative position and orientation between objects — tasks where language priors are least useful. Achieving double-digit absolute improvements on both of these "anti-prior" benchmarks represents SLVR's most compelling experimental evidence, directly supporting its central claim: forcing the model to rely on image evidence genuinely improves visual reasoning ability rather than merely inflating aggregate scores. The fact that Qwen2.5-VL-7B — already one of the stronger open-source MLLMs for visual understanding — still yields significant gains on top of this foundation suggests that SLVR's training paradigm changes the reasoning mechanism itself, rather than simply compensating for weaknesses in the base model.

Significance and Outlook

SLVR's value lies in its discovery of a balance point between explicit and latent reasoning. Explicit chain-of-thought is interpretable but expensive; pure latent reasoning is efficient but uncontrolled. SLVR combines the strengths of both through "typed latent stages."

This "human-like reasoning flow" philosophy — plan first, then ground, select evidence, and finally integrate — also echoes cognitive science's understanding of how humans process visual reasoning tasks. For practical applications seeking a tradeoff between deployment cost and reasoning capability, this type of approach offers a valuable reference direction.

Of course, as a newly released preprint, SLVR's generalization capability, performance on larger-scale models, and adaptability to different types of visual tasks all remain to be further validated. Interested readers can visit the project page for more details.