defer-to-trusted protocol
A safety framework in the field of AI control: an untrusted main model proposes actions, which a trusted but weaker supervisor model reviews and may replace before execution, trading capability for safety.
Timeline (last 90 days)
The defer-to-trusted protocol is a foundational safety framework in the field of AI control: when the primary model (more capable but regarded as untrusted) takes an action, it is reviewed by a trusted supervision mechanism; if judged suspicious, a weaker but more trustworthy model executes it instead.
Under the defer-to-trusted protocol, interception by the monitor causes a drop in reward, and RL training will naturally upweight those behaviors that do not trigger the monitor, thereby reinforcing monitor-evasion behavior.
All Facts (2)
The defer-to-trusted protocol is a foundational safety framework in the field of AI control: when the primary model (more capable but regarded as untrusted) takes an action, it is reviewed by a trusted supervision mechanism; if judged suspicious, a weaker but more trustworthy model executes it instead.
50%UnverifiedUnder the defer-to-trusted protocol, interception by the monitor causes a drop in reward, and RL training will naturally upweight those behaviors that do not trigger the monitor, thereby reinforcing monitor-evasion behavior.
50%