#SAE
3 related articles
Sparse Coding Innovation: How ConSDL Achieves Cross-Training Consistent Concept Discovery
SAE dictionaries fluctuate across training runs. New research ConSDL replaces the linear encoder with per-sample sparse coding, achieving near-perfect dictionary recovery in 16 synthetic settings and improving dictionary and coefficient consistency by 17.6% and 35.9% respectively on real tasks.
Neural Networks Are Already "Clean": A Paradigm Shift in Interpretability Research
Interpretability research is undergoing a paradigm shift: the author argues neural networks are already "clean," and the mess comes from how we observe them, not from model structure. This article analyzes the divide between old and new paradigms, its impact on SAEs and circuit analysis, and how practitioners should adjust their debugging mindset.
What Do Sparse Autoencoder Feature Discovery Scores Actually Identify? The Gap from Activations to Interventions
Can SAE feature discovery scores predict intervention effects? New research says no: probe-to-attribution ratios reach hundreds, gradient sign agreement 91-97% vs. probes at 17-35%.