Neighborhood Overlap
A local, non-parametric criterion for representational alignment that measures whether the nearest-neighbor set of each sample is consistent between two sets of representations; it can remain high even when global structure differs entirely.
Timeline (last 90 days)
Subsequent studies using different means such as neighborhoods, spanning trees, and distances to test representational alignment have reached mutually contradictory conclusions.
In the lattice structure, the convergence claimed by PRH is located at the top of the lattice and is a global, strong-sense representational consistency; the empirical evidence supporting this claim often comes from neighborhood overlap, which sits near the bottom of the lattice and is a much weaker, local criterion.
The paper points out that calibration causes neighborhood overlap to continue to grow as model scale increases, yet essentially eliminates the growth of centered kernel alignment (CKA).
Neighborhood overlap only concerns whether the nearest-neighbor set of each sample is consistent between the two representations, and is a purely local, nonparametric criterion.
Neighborhood overlap can remain high even when the global structures are completely different, as long as local proximity relations are preserved; whereas a significant increase in CKA usually implies alignment at a more global level.
All Facts (5)
Subsequent studies using different means such as neighborhoods, spanning trees, and distances to test representational alignment have reached mutually contradictory conclusions.
50%UnverifiedIn the lattice structure, the convergence claimed by PRH is located at the top of the lattice and is a global, strong-sense representational consistency; the empirical evidence supporting this claim often comes from neighborhood overlap, which sits near the bottom of the lattice and is a much weaker, local criterion.
50%UnverifiedThe paper points out that calibration causes neighborhood overlap to continue to grow as model scale increases, yet essentially eliminates the growth of centered kernel alignment (CKA).
50%UnverifiedNeighborhood overlap only concerns whether the nearest-neighbor set of each sample is consistent between the two representations, and is a purely local, nonparametric criterion.
50%UnverifiedNeighborhood overlap can remain high even when the global structures are completely different, as long as local proximity relations are preserved; whereas a significant increase in CKA usually implies alignment at a more global level.
50%