NLA
A class of meta-models that map the internal activations of a language model to natural-language explanations. The generated explanations are natural-language descriptions, which cannot be directly verified through forward propagation.
Timeline (last 90 days)
On held-out activation directions, MAEM's inversion performance is better than corpus search and the natural language autoencoder (NLA).
In identifying the concepts behind steering vectors, MAEM's performance is roughly on par with NLA.
Unlike the explanations generated by NLA, which are difficult to verify directly, the text examples generated by MAEM allow activation similarity to be computed via a single forward pass through the subject model, thereby yielding an objective verification score.
A natural language autoencoder (NLA) is a type of model that maps a model's internal activation vectors to natural language explanations and can reconstruct the activation back from that explanation.
NLA training uses an "explain-reconstruct" loop objective: given the activation of a certain layer, generate a textual description, then reconstruct the original activation from this text alone, optimizing the fidelity of the explanation with a reconstruction loss.
The core of VERS is to use a natural language autoencoder (NLA) to produce textual explanations of and reconstruct model activations, and to steer the reasoning process by editing the explanations.
All Facts (6)
On held-out activation directions, MAEM's inversion performance is better than corpus search and the natural language autoencoder (NLA).
50%UnverifiedIn identifying the concepts behind steering vectors, MAEM's performance is roughly on par with NLA.
50%UnverifiedUnlike the explanations generated by NLA, which are difficult to verify directly, the text examples generated by MAEM allow activation similarity to be computed via a single forward pass through the subject model, thereby yielding an objective verification score.
50%UnverifiedA natural language autoencoder (NLA) is a type of model that maps a model's internal activation vectors to natural language explanations and can reconstruct the activation back from that explanation.
50%UnverifiedNLA training uses an "explain-reconstruct" loop objective: given the activation of a certain layer, generate a textual description, then reconstruct the original activation from this text alone, optimizing the fidelity of the explanation with a reconstruction loss.
50%UnverifiedThe core of VERS is to use a natural language autoencoder (NLA) to produce textual explanations of and reconstruct model activations, and to steer the reasoning process by editing the explanations.
50%