From Plato to Aristotle: Toward a General Theory of Representation Hypotheses

The field of representation learning has long been troubled by an important question: to what extent are the internal representations learned by different models actually shared? The Platonic Representation Hypothesis (PRH) offers an optimistic answer—as model capabilities improve, they will converge toward a common representational structure. But subsequent studies examining representational alignment using different tools—neighborhoods, spanning trees, distances—have reached contradictory conclusions. This new work proposes the Lattice Representation Hypothesis (LRH), attempting to provide a unified explanatory framework for these disagreements, and further introduces the Aristotelian Representation Hypothesis (ARH) to characterize local convergence phenomena.

Where Do the Disagreements Come From: Alignment Criteria Occupy Different Positions in the Same Lattice

The paper's core insight is that those seemingly contradictory alignment conclusions do not actually conflict, because the alignment criteria behind them fall at different positions within the same lattice.

In this lattice structure, the convergence claimed by PRH sits at the top—a global, strong sense of representational consistency. The empirical evidence supporting this claim often comes from neighbor overlap, which sits near the bottom of the lattice—a much weaker, local criterion. In other words, there is a mismatch between the strength of the conclusion and the strength of the evidence: using a bottom-level criterion to argue for a top-level conclusion leaves a missing bridge in between.

LRH formalizes this mismatch, so that disagreements between studies are no longer simple disputes over right and wrong, but become differences that can be located and compared. The authors hope to thereby provide an analytical framework for various representation hypotheses, allowing their evidence to be compared within the same coordinate system.

The Mathematical Meaning of a Lattice

In order theory, a lattice is an algebraic structure with a partial order, in which any two elements have a unique least upper bound (join) and greatest lower bound (meet). It is often drawn as a Hasse diagram: the higher up, the "larger" and stronger the element; the lower down, the "smaller" and weaker. This article borrows this structure to organize representation hypotheses—not to say that representations themselves constitute a lattice in the strict mathematical sense, but to emphasize that different alignment criteria have a partial order of strength: top-level criteria logically imply bottom-level criteria, but not vice versa. Thus, within the lattice's coordinate system, one can ask which level of properties a given piece of experimental evidence actually measures, and can judge whether additional assumptions are needed to move from one criterion to another. The benefit of this formalization is that it transforms vague "conflicting conclusions" into locatable "level mismatches," allowing the conclusions of different papers to be compared on the same diagram.

The Aristotelian Representation Hypothesis: Convergence Occurs Only Locally

If PRH is Platonic—pursuing a common representational ideal beyond individuals—then the Aristotelian Representation Hypothesis (ARH) confines convergence to local neighborhoods.

The paper points out that calibration, as model scale grows, allows neighbor overlap to continue increasing, yet essentially eliminates the growth of centered kernel alignment (CKA). This means that evidence for convergence at the global level is weakened, while similarity at the local level still persists. The authors further prove that this elimination effect can arise from a calibration operation that takes the maximum across multiple layers, so whether global convergence holds remains an open question.

This distinction is quite critical: it suggests that the "similarity" between models may only be similarity in local geometric structure, and may not generalize to consistency across the entire representation space.

The Difference Between Centered Kernel Alignment (CKA) and Neighbor Overlap

CKA is a class of metrics measuring the overall similarity of two representation spaces. Its core idea is to first center the features, then compare the correspondence between sample similarity matrices, making it insensitive to global linear transformations and reflecting the geometric consistency of the entire representation space. Neighbor overlap, by contrast, only cares about whether each sample's nearest-neighbor set is consistent between two sets of representations—a purely local, non-parametric criterion. The key difference between the two is that neighbor overlap can remain high even when the global structures are completely different, as long as local proximity relations are preserved; while a significant increase in CKA usually implies alignment at a more global level. Calibration, through operations such as taking the maximum across multi-layer representations, reshapes the local distance scale, making neighborhood relations more stable while potentially disrupting the overall correspondence between sample similarity matrices—precisely explaining the phenomenon in which neighbor overlap grows while CKA growth is eliminated.

Re-examining Existing Evidence

The paper also offers reinterpretations of several specific methods.

Back into Plato's Cave (BPC) infers a coarse shared structure from the collapse of neighbor overlap on a larger evaluation set, but the authors argue that this stems at least in part from a more mundane mechanism: a fixed number of neighbors covers a shorter distance on a larger set, thereby artificially lowering the overlap.

The spanning tree overlap used by Structure over Geometry (SOG) largely just repeats the information of one-nearest-neighbor overlap, and does not provide new evidence independent of the neighborhood criterion.

The common thrust of these analyses is that many metrics treated as evidence for global convergence actually measure properties near the bottom of the lattice.

Extending to Independently Trained Models, with Theoretical and Experimental Support

LRH's scope of applicability is not limited to models under the same training pipeline. The paper extends it to independently trained models and proposes that the training pipeline and training objective determine which criteria can stably hold across multiple runs.

The authors provide proofs of this claim in linear autoencoders for the training pipeline, and in finite contrastive Gibbs models for the training objective. Experimentally, the study covers dozens of pretrained vision and language models, as well as thousands of runs of two theoretical models, with conclusions consistently supported.

Linear Autoencoders and Contrastive Gibbs Models as Theoretical Probes

A linear autoencoder is one of the simplest representation learning models, whose encoder and decoder are both linear mappings, and whose training objective is to minimize reconstruction error. Because of its simple structure, its optimal solution can be obtained analytically, so it is often used to study how the training pipeline (e.g., initialization, optimization path) affects the geometric properties of representations. The finite contrastive Gibbs model is a type of energy model that learns representations by contrasting statistics of positive and negative samples, and the training objective directly determines which distribution the model converges to. The authors' choice of these two models is not accidental: the former isolates the variable of "how to train," while the latter isolates the variable of "what to train," thereby respectively proving that the training pipeline and training objective determine which alignment criteria can stably hold across multiple independent runs. This combination of "theoretical models plus large pretrained models" gives the conclusions both provable rigor and generalizability to real large-scale models.

Significance and Limitations

The greatest value of this work lies in transforming a question prone to positional disputes into an analyzable structural question. Through the lattice framework, different representation hypotheses can be located within a unified coordinate system, and the strength of their evidence and the scope they cover become clear accordingly.

It is important to remain clear-headed: the paper itself also acknowledges that global convergence remains open. LRH provides an analytical tool and comparative framework, not a final verdict on PRH. For researchers concerned with representational alignment, model interpretability, and cross-model transfer, this may be a starting point for re-examining existing evidence.

One-line Summary

The Lattice Representation Hypothesis provides a unified analytical framework for disagreements in representational alignment research, pointing out a level mismatch between convergence conclusions and evidence.

Paragraph Summary

This new work proposes the Lattice Representation Hypothesis (LRH), attempting to explain seemingly contradictory conclusions in representational alignment research. The paper's core insight is that the alignment criteria of different studies actually fall at different positions in the same lattice—the global convergence claimed by PRH sits at the top of the lattice, while the neighbor overlap evidence supporting it sits at the bottom, creating a mismatch between the strength of the conclusion and the strength of the evidence. The authors further introduce the Aristotelian Representation Hypothesis (ARH), pointing out that calibration preserves the growth of neighbor overlap while essentially eliminating the growth of centered kernel alignment, leaving global convergence an open question. The paper also reinterprets evidence from methods such as BPC and SOG, arguing that they actually measure properties near the bottom of the lattice. LRH can be extended to independently trained models and is theoretically proven on linear autoencoders and finite contrastive Gibbs models, with experiments covering dozens of pretrained vision and language models.