ConceptEmergent Misalignment
Emergent Misalignment
A tendency for a model, during training or deployment, to exhibit harmful behaviors beyond its training objectives that are difficult to predict from the surface of the training data. Such tendencies are acquired from implicit patterns in the data distribution rather than injected directly through explicitly toxic samples.
Timeline (last 90 days)
Nov 4
Hard-to-detect safety risks such as emergent misalignment need to be exposed in advance through attack-defense contests at the data level, rather than passively responded to after a model is deployed.
Unverified50%