Sparse Autoencoders
An activation-reading method that trains an overcomplete sparse dictionary on model activations. An encoder maps the high-dimensional residual stream into sparse features, each corresponding to a nameable concept, and can output multi-token natural-language descriptions.
Core Facts
Timeline (last 90 days)
SAE still faces challenges: it is sensitive to hyperparameters such as dictionary size and sparsity strength, the extracted features may not cover all of the model's computation, and sometimes it merely transfers polysemy to another layer of representation without thoroughly eliminating it at the causal level.
The activation-reading tools evaluated are divided into single-token readers and multi-token readers: the single-token side includes Logit lens, J-lens, R-lens, and Tuned lens; the multi-token side includes Natural Language Autoencoders, Sparse Autoencoders, Patchscopes, Template lens, and Oracle Lens.
The Anthropic team used sparse autoencoders (SAE) to extract millions of interpretable features from model activations.
All Facts (4)
Sparse autoencoders (SAEs) are one of the main technical approaches attempting to disentangle superposition and extract monosemantic features.
80%UnverifiedSAE still faces challenges: it is sensitive to hyperparameters such as dictionary size and sparsity strength, the extracted features may not cover all of the model's computation, and sometimes it merely transfers polysemy to another layer of representation without thoroughly eliminating it at the causal level.
50%UnverifiedThe activation-reading tools evaluated are divided into single-token readers and multi-token readers: the single-token side includes Logit lens, J-lens, R-lens, and Tuned lens; the multi-token side includes Natural Language Autoencoders, Sparse Autoencoders, Patchscopes, Template lens, and Oracle Lens.
50%UnverifiedThe Anthropic team used sparse autoencoders (SAE) to extract millions of interpretable features from model activations.
50%