#dead latents

HyperSAE: How Hyperbolic Geometry Solves the Dead Latent Problem in Sparse Autoencoders

HyperSAE uses Poincaré ball hyperbolic geometry to replace Euclidean space in sparse autoencoders, reducing dead latents from 3.8% to 0.2% with zero inference cost.