SAE PII Suppression Fails: Logprob-Only Evaluation Misleading

Sparse autoencoders (SAEs) are often used to intervene at the feature level in large models to reduce the leakage of sensitive personal information (PII). Existing evaluations almost exclusively look at one metric: whether the log-probability of the target token decreases after intervention. A new study directly challenges this default practice, pointing out that a logprob drop may simply be an artifact of "perturbation magnitude" rather than truly severing the causal pathway of PII leakage.

Key Finding: 17 "Successful" Cases, Only 1 Withstands Multi-Metric Testing

The study covers 36 (model, layer, PII type) combinations, comparing selected SAE feature interventions against a control group of "coherence-matched random directions." Evaluation uses not only logprob but also rank-aware metrics.

The results are striking: when looking only at logprob, 17 cases (47%) appear to have successfully suppressed PII; but when switched to a multi-metric framework, only 1 case (2.8%) passes. In other words, nearly half of the "effective interventions" do not hold up under stricter testing.

This points to a methodological problem: a logprob drop itself cannot distinguish between "feature-specific suppression" and "magnitude artifacts"—as long as the activation perturbation is large enough, it does not matter whether the direction is aligned with the target feature.

Rank-aware metrics refer to examining not only whether the absolute probability or log-probability of the target token decreases, but also the change in the token's relative ranking within the candidate vocabulary. For example, if the target PII token was originally ranked 10th and drops to 100th after intervention, even if its logprob drop is modest, this indicates directional suppression; conversely, if the logprob drops but the ranking remains unchanged, it is more likely a magnitude artifact caused by flattening the overall distribution. Such metrics can partially distinguish "precise suppression" from "perturbation magnitude" and are a key component of multi-metric evaluation frameworks.

Two Failure Modes: Random Directions Outperform, Feature Selection Collapses

Among the cases that fail multi-metric testing, the study identifies two typical failure modes.

The first is random directions outperforming selected features. When interventions with random direction controls outperform carefully selected SAE features, it suggests that the logprob drop likely comes from perturbation magnitude rather than the semantic role of the feature itself.

The second is feature selection collapse across PII types. Features selected for one type of PII fail on another type, suggesting that these features do not lock onto the general causal mechanism of PII leakage.

Together, these two failure modes indicate that existing evaluation pipelines cannot verify whether an intervention truly targets the causal pathway of PII leakage.

9,000 Causal Mediation Validations: Most Correlation-Selected Candidates Fail

For further testing, the study also conducted a causal mediation feasibility analysis with 9,000 runs. The conclusion: the validator itself can reliably distinguish candidate interventions, but candidates selected through correlation screening rarely pass this validation.

This serves as a warning for the current pipeline of "select features—lower logprob—declare success." Correlation screening may select features that are superficially related to PII leakage but are not necessarily key nodes in the causal chain.

Causal mediation analysis is a statistical framework used to determine whether an intermediate variable (here, SAE feature activation) mediates the causal pathway between input (a prompt containing PII) and output (PII leakage). A typical approach is to intervene on the mediator variable and observe changes in the outcome while controlling for confounders. If a feature is merely correlated with leakage rather than a causal mediator, intervening on it will not stably block leakage. This analysis can distinguish "superficially correlated features" from "key nodes in the causal chain" and is the core of the evaluation the authors advocate.

Research Initiative: From Correlation Evaluation to Causal Localization

Based on these findings, the authors propose that the NLP field needs a new research task: causally grounded PII suppression, i.e., using circuit-level localization and causal mediation analysis to confirm whether an intervention acts on the true leakage pathway.

For practitioners, this means two things. First, reporting only a logprob drop is insufficient to support the conclusion that "SAE features effectively suppress PII"; at minimum, direction-matched random controls and rank-aware metrics should be added. Second, the feature selection step needs causal validation rather than relying solely on correlation ranking.

The value of this work is not to deny the potential of SAEs, but to point out that evaluation without causal validation may systematically overestimate the effectiveness of feature-level interventions.

Circuit-level localization refers to identifying, within a neural network, the minimal computational subgraph that implements a specific function (such as PII leakage), including attention heads, MLP neurons, and their connections. Unlike localizing a single feature, it emphasizes causal interactions among features. In the context of PII suppression, circuit-level localization can help confirm whether an intervention acts on the real leakage pathway rather than merely perturbing isolated activations. This concept originates from mechanistic interpretability research and is being introduced into safety intervention evaluation.

One-line Summary

Looking only at logprob drops overestimates the effect of SAE features in suppressing PII; a shift to causal validation is needed.

Paragraph Summary

A new study challenges the existing evaluation of sparse autoencoders (SAEs) for suppressing PII leakage in large models. Covering 36 (model, layer, PII type) combinations, it finds that when looking only at logprob drops, 47% of cases appear successful, but after adding rank-aware metrics and random-direction controls, only 2.8% pass. Failure modes include random directions outperforming selected features and feature selection collapse across PII types, indicating that logprob drops may simply be perturbation magnitude artifacts. 9,000 causal mediation validations further show that candidates selected by correlation rarely pass causal tests. The authors advocate a shift toward circuit-level localization and causal mediation analysis to establish a causally grounded evaluation framework.