Contrastive Activation Probing
A safety technique for detecting latent behavioral couplings in language models. It identifies potentially amplified harmful behavior pathways by contrasting the directions of activation change in a model before and after fine-tuning.
Timeline (last 90 days)
The paper's authors suggest integrating contrastive direction probing into DevSecOps artifact verification pipelines, comparing the directional deviation of intermediate-layer activations between the base model and the fine-tuned model under probe prompts, rather than only looking at output behavior.
Contrastive activation probing uses cosine similarity to measure the directional deviation of intermediate-layer activations between the base model and the fine-tuned model under the same probe prompts, with lower similarity indicating greater drift.
All Facts (2)
The paper's authors suggest integrating contrastive direction probing into DevSecOps artifact verification pipelines, comparing the directional deviation of intermediate-layer activations between the base model and the fine-tuned model under probe prompts, rather than only looking at output behavior.
50%UnverifiedContrastive activation probing uses cosine similarity to measure the directional deviation of intermediate-layer activations between the base model and the fine-tuned model under the same probe prompts, with lower similarity indicating greater drift.
50%