Misalignment
A core concept in AI safety referring to a model's behavior, goals or outputs deviating from human intentions and value norms, as distinct from mere lack of capability.
Timeline (last 90 days)
Fine-tuning a large language model on data that the model itself considers wrong but whose content is harmless can induce broad misalignment.
On math tasks, as the training answers become increasingly absurd, the proportion of misaligned responses soars from 4% to over 60%, while the model's general capabilities show almost no loss.
Misalignment is not a byproduct of capability degradation, but an independent behavioral deviation.
When a false fact is packaged as part of a coherent story, the model believes it more deeply, and the degree of misalignment is lower than when the model is directly instructed to assert that false fact.
All Facts (4)
Fine-tuning a large language model on data that the model itself considers wrong but whose content is harmless can induce broad misalignment.
50%UnverifiedOn math tasks, as the training answers become increasingly absurd, the proportion of misaligned responses soars from 4% to over 60%, while the model's general capabilities show almost no loss.
50%UnverifiedMisalignment is not a byproduct of capability degradation, but an independent behavioral deviation.
50%UnverifiedWhen a false fact is packaged as part of a coherent story, the model believes it more deeply, and the degree of misalignment is lower than when the model is directly instructed to assert that false fact.
50%