Jacobian Lens
A model-specific component from Anthropic's global workspace paper that captures directions of a model's internal states. It forms part of the mechanism through which U-Lens is implemented.
Timeline (last 90 days)
Latent-space reasoning architectures may also make existing interpretability techniques harder to use; for example, Jacobian Lens relies on the model converting thoughts into token probabilities, and the interpretation of SAEs relies on token input-output relationships — if only a very small portion of the model's cognition passes through tokens, the reliability of these techniques will all decline.
The only model-specific component U-Lens relies on is the Jacobian Lens (J-Lens), from Anthropic's global workspace paper.
J-Lens estimates the Jacobian matrix from the hidden state to the output space, linearly mapping the hidden state of any layer back to an interpretable token space.
J-Lens is model-specific but task-agnostic: once fitted for a certain model, it can be reused for any prompt without retraining.
To fit J-Lens yourself, you can use Anthropic's reference code (anthropics/jacobian-lens, jlens.fit), which requires about 100 unlabeled prompts, trains no parameters, and whose cost mainly comes from backpropagation.
All Facts (5)
Latent-space reasoning architectures may also make existing interpretability techniques harder to use; for example, Jacobian Lens relies on the model converting thoughts into token probabilities, and the interpretation of SAEs relies on token input-output relationships — if only a very small portion of the model's cognition passes through tokens, the reliability of these techniques will all decline.
50%UnverifiedThe only model-specific component U-Lens relies on is the Jacobian Lens (J-Lens), from Anthropic's global workspace paper.
50%UnverifiedJ-Lens estimates the Jacobian matrix from the hidden state to the output space, linearly mapping the hidden state of any layer back to an interpretable token space.
50%UnverifiedJ-Lens is model-specific but task-agnostic: once fitted for a certain model, it can be reused for any prompt without retraining.
50%UnverifiedTo fit J-Lens yourself, you can use Anthropic's reference code (anthropics/jacobian-lens, jlens.fit), which requires about 100 unlabeled prompts, trains no parameters, and whose cost mainly comes from backpropagation.
50%