Intervention Attribution
An attribution idea drawn from causal inference that measures the first-order change in model output after applying a specific intervention to an internal variable, and can be approximated as the partial derivative of the output with respect to that latent variable.
Timeline (last 90 days)
For a large class of response functions, observational feature discovery evidence generally cannot identify intervention attribution, which is a structural conclusion rather than a flaw of any specific method.
Intervention attribution itself is gradient-estimable, and gradient information can be used to approximate the impact of intervention features on the model output, without actually running intervention experiments.
In measurements across four model-SAE pairings, the ratio of probe attribution to intervention attribution ranged from tens to hundreds at the median, with probes systematically overestimating the intervention efficacy of features.
The feature set selected using intervention attribution produced a larger absolute mean ablation effect on held-out prompts than the sets selected by correlation selection, rank-test selection, and probe-coefficient selection.
Gradient estimation is valid only when the intervention magnitude is sufficiently small and the model is locally approximately linear; for large-scale steering or highly nonlinear response functions, the first-order approximation may be distorted.
The paper recommends establishing intervention attribution as the principled objective for selecting SAE features for intervention, and this objective can be estimated via gradients.
The gradient decomposition given in the paper breaks down the gap between linear probe predictions and true intervention attribution, showing that intervention attribution can be estimated via gradients.
All Facts (7)
For a large class of response functions, observational feature discovery evidence generally cannot identify intervention attribution, which is a structural conclusion rather than a flaw of any specific method.
50%UnverifiedIntervention attribution itself is gradient-estimable, and gradient information can be used to approximate the impact of intervention features on the model output, without actually running intervention experiments.
50%UnverifiedIn measurements across four model-SAE pairings, the ratio of probe attribution to intervention attribution ranged from tens to hundreds at the median, with probes systematically overestimating the intervention efficacy of features.
50%UnverifiedThe feature set selected using intervention attribution produced a larger absolute mean ablation effect on held-out prompts than the sets selected by correlation selection, rank-test selection, and probe-coefficient selection.
50%UnverifiedGradient estimation is valid only when the intervention magnitude is sufficiently small and the model is locally approximately linear; for large-scale steering or highly nonlinear response functions, the first-order approximation may be distorted.
50%UnverifiedThe paper recommends establishing intervention attribution as the principled objective for selecting SAE features for intervention, and this objective can be estimated via gradients.
50%UnverifiedThe gradient decomposition given in the paper breaks down the gap between linear probe predictions and true intervention attribution, showing that intervention attribution can be estimated via gradients.
50%