Natural Language Autoencoders
An activation-reading method in which an encoder compresses activations into a latent vector and a language-model decoder generates a natural-language description. It offers the strongest expressiveness but may produce content that does not match the facts.
Timeline (last 90 days)
The activation-reading tools evaluated are divided into single-token readers and multi-token readers: the single-token side includes Logit lens, J-lens, R-lens, and Tuned lens; the multi-token side includes Natural Language Autoencoders, Sparse Autoencoders, Patchscopes, Template lens, and Oracle Lens.
Natural Language Autoencoders use an encoder to compress activations into latent vectors, then a decoder generates natural language descriptions; the training objective includes reconstructing the activations, giving it the strongest expressiveness and the longest readout length, but the training signal does not force the descriptions to be consistent with the facts, so it easily generates content that looks plausible yet contradicts the actual output.
All Facts (2)
The activation-reading tools evaluated are divided into single-token readers and multi-token readers: the single-token side includes Logit lens, J-lens, R-lens, and Tuned lens; the multi-token side includes Natural Language Autoencoders, Sparse Autoencoders, Patchscopes, Template lens, and Oracle Lens.
50%UnverifiedNatural Language Autoencoders use an encoder to compress activations into latent vectors, then a decoder generates natural language descriptions; the training objective includes reconstructing the activations, giving it the strongest expressiveness and the longest readout length, but the training signal does not force the descriptions to be consistent with the facts, so it easily generates content that looks plausible yet contradicts the actual output.
50%