Linear Response Map
A neural network analysis tool inspired by the Green's function concept from physics. It characterizes causal transmission relationships among model components through first-order linear approximation, and is used to quantify the influence of task context on attention head interaction patterns.
Timeline (last 90 days)
The researchers construct two linear response graphs before and after activation editing for each prompt, used to predict the response of later attention heads based on measured changes in earlier attention heads.
Compared to the average of the two graphs, the conditioned response graph reduces mean squared prediction error by 53.6% on Gemma-2-9B.
The linear response graph draws on the Green's function concept from physics and neuroscience, using a first-order linear approximation to predict the change in response of downstream components to perturbations in earlier components.
The generalizability of the linear response framework still needs to be validated on more models and tasks.
All Facts (4)
The researchers construct two linear response graphs before and after activation editing for each prompt, used to predict the response of later attention heads based on measured changes in earlier attention heads.
50%UnverifiedCompared to the average of the two graphs, the conditioned response graph reduces mean squared prediction error by 53.6% on Gemma-2-9B.
50%UnverifiedThe linear response graph draws on the Green's function concept from physics and neuroscience, using a first-order linear approximation to predict the change in response of downstream components to perturbations in earlier components.
50%UnverifiedThe generalizability of the linear response framework still needs to be validated on more models and tasks.
50%