How Do Vision Transformers Understand Abstract Concepts? The Metonymic Circuit Mechanism Explained

The Challenge of Recognizing Abstract Concepts

Vision Transformers excel at image recognition, but a long-standing question remains: how do these models understand abstract concepts like "angry"? Unlike concrete objects such as "cat" or "car," abstract concepts rarely have direct visual referents in training data. There is no "anger" entity in a picture — so what does the model actually rely on to make such judgments?

A newly published arXiv paper, Metonymic Circuits for Abstract Concept Grounding in Vision Transformers, offers a compelling answer: models may leverage a metonymic grounding mechanism, using concrete, interpretable "anchor concepts" to bridge the gap between visual signals and abstract semantics.

What Is Metonymic Grounding?

The paper's core hypothesis is that when a model needs to predict an abstract concept, its judgment is actually driven by a series of concrete, interpretable intermediate concepts. For example, recognizing "anger" might depend on a tangible element like "fire" — something visually perceptible that also carries a metaphorical association with anger.

This mechanism is known in linguistics as "metonymy" — using a related, concrete entity to refer to an abstract meaning. The researchers transplant this concept into the internals of vision models, seeking to verify whether models unconsciously employ a similar strategy: first locking onto concrete visual anchors, then using those anchors to reach the abstract target.

Why This Perspective Matters

Traditional interpretability research has largely focused on how models recognize concrete objects, while the internal representations of abstract concepts have remained a black box within a black box. The metonymic grounding framework offers an actionable path for researchers to trace the "reasoning chain" behind abstract judgments, rather than treating them as an indivisible whole.

CLIP (Contrastive Language-Image Pre-Training) and DINO are two of the most widely used visual encoders today, representing fundamentally different training paradigms. CLIP, developed by OpenAI, uses contrastive learning between images and natural language descriptions to naturally align image embedding spaces with textual semantics — making it particularly effective for cross-modal retrieval and zero-shot recognition. Its internal features are considered rich in high-level semantic information, though this also makes it harder to trace the contributions of specific visual signals. DINO (and its successor DINOv2), on the other hand, is trained via self-supervised distillation without text pairings, yet learns visual features with strong spatial localization — attention maps often clearly delineate object boundaries. Studying both encoders within the same framework helps determine whether metonymic circuits are a universal mechanism in vision models or an emergent phenomenon specific to certain training objectives.

Dissecting Internal Circuits with Transcoder

To test their hypothesis, the research team applied Transcoder technology to both CLIP and DINO. Transcoder can recover intermediate features from a model and associate them with more concrete conceptual semantic labels, enabling the tracking of each component's contribution within the "circuit" responsible for abstract concept recognition.

The researchers also constructed a carefully curated icon dataset for their experiments. The results revealed clearly structured metonymic circuits:

• Early layers are dominated by perceptual primitives: The model's shallow layers primarily process low-level visual features such as color, shape, and texture.
• Object-level anchors emerge before abstract targets: Before reaching the abstract concept, the model first activates concrete, object-like anchor concepts.
• Text takes an independent path: When images contain rendered text, the model invokes a distinctly different "percept-to-text" routing rather than the metonymic anchor channel.

This layered structure indicates that abstract concept recognition is not instantaneous, but is built up gradually along a gradient from concrete to abstract.

Transcoder is a feature decomposition tool in mechanistic interpretability, inspired by Sparse Autoencoders (SAEs). Traditional SAEs extract interpretable features by training sparse reconstructions on the residual stream, while Transcoder specifically targets MLP layers: it trains a small "input → sparse hidden layer → output" network to replace the original MLP while preserving model behavior. This decomposes otherwise opaque dense MLP activations into sparse, semantically labeled intermediate features that researchers can directly read — identifying "which concept nodes are activated at this moment." Compared to pure activation patching or probe-based methods, Transcoder recovers a more fine-grained computational graph, turning "circuit" tracing from qualitative description into quantitative contribution attribution. In this study, it was precisely through Transcoder that the researchers were able to identify, layer by layer across CLIP and DINO, which intermediate features serve as metonymic anchors.

Causal Intervention Validates the Mechanism

Observing correlations alone is insufficient to draw conclusions. The research team further employed causal intervention methods, directly manipulating these metonymic intermediate features and observing changes in the model's final output.

The results showed that these metonymic intermediaries play a genuine functional role in abstract concept grounding — altering the activation of anchor concepts correspondingly affected the prediction of abstract concepts. This causally confirms that metonymic circuits are not merely "coincidental correlations" but actual computational pathways the model relies upon.

Causal intervention in mechanistic interpretability research is typically implemented as "activation patching" or "counterfactual substitution": researchers directly modify activation values at a specific position during the model's forward pass, then observe the magnitude of change in the final output. If suppressing an intermediate feature causes the prediction probability of an abstract concept to drop significantly, it can be inferred that this feature plays an indispensable role in the computational pathway. The key advantage of this method is its ability to distinguish "correlation" from "causation" — a model may simultaneously activate many features related to a target concept, but only through intervention can one determine which are true functional nodes and which are merely incidental byproducts. In the context of abstract concept understanding, causal intervention is particularly meaningful: it enables researchers to experimentally answer counterfactual questions like "if the model hadn't seen the anchor concept 'fire,' could it still recognize 'anger'?"

Implications for AI Interpretability Research

The value of this work lies in transforming the elusive problem of semantic grounding into tractable, verifiable internal circuit analysis. Several insights are worth noting:

First, in the absence of direct supervisory signals, vision models spontaneously use concrete concepts as "stepping stones" to understand abstract semantics — a process that bears resemblance to human cognitive metaphor mechanisms.

Second, the discovery that textual content triggers an independent processing pathway suggests functional differentiation within multimodal models, offering valuable reference for understanding how cross-modal models like CLIP work internally.

Third, the methodology combining Transcoder with causal intervention provides a reusable analytical paradigm for mechanistic interpretability research.

As the capabilities of vision and multimodal large models continue to advance, understanding why they make certain judgments becomes increasingly important. Research into metonymic circuits gives us a window into how models gradually elevate pixel signals into abstract semantics — carrying long-term significance for building more transparent and controllable AI systems.