Representation Engineering
A research direction in large language model studies that investigates the linear representational structure of high-level semantic concepts—such as emotion or moral tendency—within a model's activation space. It involves identifying and manipulating these directional vectors to influence model behavior.
Timeline (last 90 days)
The results of this paper are in tension with the assumption of the RepE framework by Zou et al. 2023, which seeks stable control vectors across contexts.
Activation steering or representation engineering applies intervention vectors directly to the model's internal activation layers, altering the generation direction without modifying the model's weights.
The theoretical foundation of abliteration comes from the field of representation engineering, which has found that certain high-level semantic concepts within a model can be identified as specific linear directions in activation space
All Facts (3)
The results of this paper are in tension with the assumption of the RepE framework by Zou et al. 2023, which seeks stable control vectors across contexts.
50%UnverifiedActivation steering or representation engineering applies intervention vectors directly to the model's internal activation layers, altering the generation direction without modifying the model's weights.
50%UnverifiedThe theoretical foundation of abliteration comes from the field of representation engineering, which has found that certain high-level semantic concepts within a model can be identified as specific linear directions in activation space
50%