Paper
Predicting Alignment Generalization with Value Representations
A paper studying the prediction of LLM alignment generalization. It proposes using internal activation representations of a model to predict the effect of value fine-tuning on behavior in unseen situations, and constructs a large-scale generalization matrix covering 66 value dimensions.
Timeline (last 90 days)
Oct 13
The paper "Predicting Alignment Generalization with Value Representations" proposes the alignment generalization prediction task, that is, predicting the behavioral changes of a fine-tuned model on a large number of held-out values after it follows a certain value.
Unverified50%