Linear Representation Hypothesis
A hypothesis originating from mechanistic interpretability research on large language models, proposing that high-level semantic concepts correspond to linear subspaces in representation space. Vectors encoding different attributes can be linearly combined to compose meanings, and this is considered a general organizational principle in deep networks such as Transformers.
Timeline (last 90 days)
LinSlot introduces the Linear Representation Hypothesis (LRH) into the slot-based visual object representation framework.
The Linear Representation Hypothesis holds that composable concepts can be represented as linearly additive subspaces within slot representations.
The Linear Representation Hypothesis originates from mechanistic interpretability research into the internal mechanisms of large language models, such as the conceptual arithmetic 'king − male + female ≈ queen'.
All Facts (3)
LinSlot introduces the Linear Representation Hypothesis (LRH) into the slot-based visual object representation framework.
50%UnverifiedThe Linear Representation Hypothesis holds that composable concepts can be represented as linearly additive subspaces within slot representations.
50%UnverifiedThe Linear Representation Hypothesis originates from mechanistic interpretability research into the internal mechanisms of large language models, such as the conceptual arithmetic 'king − male + female ≈ queen'.
50%