Transcoder
A feature decomposition tool in mechanistic interpretability research. It works by training a sparse input-hidden-output subnetwork to replace an original MLP layer, decomposing dense activations into sparse intermediate features with semantic labels. The tool is used to trace computational circuits inside language models.
Timeline (last 90 days)
The research team applied the Transcoder technique to two visual encoders, CLIP and DINO, to verify the metonymy hypothesis.
Transcoder is a feature decomposition tool in the field of mechanistic interpretability that trains a small network specifically targeting MLP layers, going from input to a sparse hidden layer to output, to replace the original MLP.
All Facts (2)
The research team applied the Transcoder technique to two visual encoders, CLIP and DINO, to verify the metonymy hypothesis.
50%UnverifiedTranscoder is a feature decomposition tool in the field of mechanistic interpretability that trains a small network specifically targeting MLP layers, going from input to a sparse hidden layer to output, to replace the original MLP.
50%