Model Misalignment
A core concept in AI safety referring to the phenomenon in which an AI system's actual behavior deviates from developer intentions or human values. Common manifestations include reward hacking, deceptive alignment, generation of harmful content, and circumvention of safety constraints.
Timeline (last 90 days)
Model misalignment refers to the deviation between a model's actual behavior and its design objectives or human values
The academic community subdivides misalignment into levels such as goal misalignment, value misalignment, and misalignment under distributional shift
Anomalous model behavior is often difficult to fix in a clear-cut way, unlike code vulnerabilities
Reporting of model misalignment may gradually become standard practice in the AI field, much like software security vulnerability disclosure
All Facts (4)
Model misalignment refers to the deviation between a model's actual behavior and its design objectives or human values
50%UnverifiedThe academic community subdivides misalignment into levels such as goal misalignment, value misalignment, and misalignment under distributional shift
50%UnverifiedAnomalous model behavior is often difficult to fix in a clear-cut way, unlike code vulnerabilities
50%UnverifiedReporting of model misalignment may gradually become standard practice in the AI field, much like software security vulnerability disclosure
50%