[KongchangAI]
· 2 min read· 1,282 words

Why Vision-Language Models Can't Say What They See in Cross-Image Comparison

Why Vision-Language Models Can't Say What They See in Cross-Image Comparison

VLM cross-image comparison failures trace to 16-32 attention heads; training only their query/key projections yields large gains.

The study traces VLM failure mechanisms on cross-image matching across Qwen3-VL, InternVL3.5, and Gemma-3, finding two modes: comparison completed but answer not read out (probe accuracy 15-25 points above calibrated output), and semantic correspondence present in visual tokens but not passed to the answer layer. Causal intervention pinpoints 16-32 attention heads carrying cross-image comparison; redistributing their cross-image attention drops accuracy 15-32 points. Training only these heads' query/key projections (0.02% of parameters) gains 28.3 and 25.1 points on SPair-71k, beating multiple controls and offering a more economical targeted intervention than full fine-tuning.

Vision-language models (VLMs) have become quite mature at single-image understanding, but when a task requires matching a point or an object between two images, the model often fails—even when the answer is hidden in its hidden states. A new study systematically traces this failure, pinpoints two key breakpoints across three model families, and offers a fix that modifies only 0.02% of parameters.

Research source screenshot

Two Failure Modes of Cross-Image Comparison

On Qwen3-VL (2B-32B), InternVL3.5-8B, and Gemma-3-12B, the researchers traced from visual tokens all the way to the answer letter using three task types: synthetic shape matching, same-instance point correspondence (HPatches), and semantic correspondence (SPair-71k). They found that failure is not a single cause but splits into two.

The first is "compared but not read out": for identical copies, the model does perform the comparison, but the answer is not correctly read out. Probing at the answer position yields accuracy 15 to 25 points higher than calibrated output, and this holds whether or not the shapes are nameable. Steering along the probe direction, meanwhile, changes no more answers than a random direction.

The second is "information present but not passed along": the semantic correspondence exists in the visual tokens but never reaches the answer layer. This shows the problem lies in the cross-image information transfer path, not in a missing feature itself.

A probe is a common tool in mechanistic interpretability research: a simple linear classifier is trained on a model's hidden states at a given layer to determine whether that layer already encodes the information the task requires. If probe accuracy is significantly higher than the model's actual output accuracy, it means the information is already present in the hidden states and the problem lies in the readout or transfer from that layer to the final answer, not in feature extraction failure. In this article, "answer-position probe accuracy is 15 to 25 points higher than calibrated output" is a direct application of this logic, used to distinguish "didn't compute it" from "computed it but didn't say it."

Calibration refers to the consistency between a model's output probabilities and its true accuracy. An uncalibrated model may give high-confidence wrong answers, so directly comparing raw output accuracy with probe accuracy would be biased; the researchers first calibrate the outputs before comparing them with the probes, so they can fairly judge whether information is "hidden" in the hidden states.

16 to 32 Heads Carry Cross-Image Comparison

The study further pinpoints a small group of attention heads that carry the comparison function. When the cross-image attention of 16 to 32 heads is redistributed while keeping its total amount unchanged, synthetic-task accuracy drops by 17 to 22 points and HPatches by 15 to 32 points, whereas random heads or attention-matched heads cause at most a 7-point drop.

This causal evidence shows that the cross-image comparison function is highly concentrated in a few heads rather than diffused across the entire network. It also explains why full fine-tuning sometimes struggles to reliably improve cross-image tasks—the key pathway is not treated separately.

An attention head is the basic unit of the multi-head attention mechanism in a Transformer. Each layer contains multiple heads, and each head independently computes query, key, and value projections and produces a set of attention weights that determine which positions that head "reads" information from. Different heads often specialize, and prior work has shown that some heads are dedicated to specific semantic or syntactic functions. The "cross-image attention" discussed here refers to the attention connection formed when a head's queries come from one image's tokens and its keys come from another image's tokens—precisely the physical channel for information exchange between the two images.

Causal intervention here means artificially changing a specific head's attention allocation (e.g., redistributing the total amount of cross-image attention) and observing whether task accuracy changes accordingly. Unlike merely observing correlations, causal intervention can rule out confounders and confirm whether a group of heads is the function carrier. The random-head control group causes at most a 7-point drop, while the target heads cause a 15- to 32-point drop; this contrast is the core of the causal evidence.

Training Only Query and Key Projections Is Enough

The study offers a rather restrained fix: train only the query and key projections of these key heads. On SPair-71k, Qwen3-VL-8B improves by 28.3 points and InternVL3.5-8B by 25.1 points, outperforming the layer-matched-head and attention-matched-head controls, and the gains transfer to HPatches and BLINK.

In a further detail, for an unconstrained full-layer adapter, 21% to 51% of its gain is routed through the attention of the same set of heads, versus at most 7% for random heads. Moreover, this routing works by changing what these heads "read" (accounting for 83% to 86% of the effect), not by changing their weights.

Query and key projections are the two linear transformation matrices inside an attention head: the query projection determines "what I'm looking for," and the key projection determines "what I can offer"; together they determine the attention weights, i.e., information routing. Training only these two projections without touching the value and output projections amounts to adjusting only "who should attend to whom" without changing the representation of the content being attended to. This explains why the fix uses so few parameters (0.02%) yet significantly improves cross-image tasks: the bottleneck is routing, not feature quality.

Layer-matched heads and attention-matched heads are control designs: the former selects heads in the same layer as the target heads but with different functions, while the latter selects heads with similar attention patterns but outside the target group. Neither matches the target heads' gains, showing that the effect does not come from "training the same number of parameters" or "training heads with similar attention," but specifically from the set of heads that carry the cross-image comparison function.

Implications for Multimodal Research

The methodology of this work also deserves attention: every decision rule was frozen before running, and all reported numbers were recomputed by an independent verification script. This kind of pre-registration rigor is uncommon in model-mechanism research.

From an application perspective, this means improving cross-image comparison may not require large-scale retraining. Pinpointing the attention heads that carry the function and applying a very small targeted intervention can yield significant gains. For products that need image-text comparison, document comparison, or visual grounding, this path offers a more economical approach than full fine-tuning.

Of course, the current conclusions rest on specific model families and task sets, and whether they generalize to more complex scenarios such as video and multi-image reasoning still needs further validation. But dissecting the mechanism of the "see it but can't say it" phenomenon down to the attention-head level is itself an important step toward understanding how multimodal models work internally.

One-line Summary

VLM cross-image comparison failures can be traced to 16-32 attention heads; training only their query and key projections yields large gains.

Paragraph Summary

This study systematically traces the failure mechanisms of vision-language models on cross-image matching tasks, identifying two failure modes on Qwen3-VL, InternVL3.5, and Gemma-3: first, the model completes the comparison but the answer is not correctly read out, with probe accuracy 15 to 25 points above calibrated output; second, semantic correspondence information exists in visual tokens but is not passed to the answer layer. Through causal intervention, it further pinpoints that 16 to 32 attention heads carry the cross-image comparison function, and redistributing their cross-image attention causes a 15- to 32-point accuracy drop. Based on this, training only the query and key projections of these heads (0.02% of parameters) yields 28.3- and 25.1-point gains on SPair-71k, outperforming multiple control groups and offering a more economical targeted intervention path than full fine-tuning for cross-image tasks.

#vision-language models#cross-image comparison#attention heads#mechanistic interpretability#SPair-71k#multimodal models#query and key projections
Share:

Related articles