Instruction Hierarchy
A safety alignment framework proposed by OpenAI and other institutions in which instructions from different sources carry different trust levels; when instructions conflict, the model must prioritize higher-priority instructions in order to prevent prompt injection attacks.
Timeline (last 90 days)
The document reveals that model behavior is governed jointly by multi-level instructions, with system-level instructions, developer instructions, user instructions, and low-trust material from documents, web pages and tool results in descending order of priority.
When instructions conflict, higher-priority provisions win; user preferences for format, tone and content can significantly influence responses but cannot override higher-level constraints.
Defenses against prompt injection (classifiers, instruction hierarchies, source labels) are without exception probabilistic; the model may be bypassed by novel phrasing, and classifiers may misjudge legitimate input.
The fundamental logic of defending against indirect injection is the separation of data and instructions: retrieved content should be confined to a read-only data zone, and hierarchical isolation can be established by placing external content in the tool role rather than the user role.
All Facts (4)
The document reveals that model behavior is governed jointly by multi-level instructions, with system-level instructions, developer instructions, user instructions, and low-trust material from documents, web pages and tool results in descending order of priority.
50%UnverifiedWhen instructions conflict, higher-priority provisions win; user preferences for format, tone and content can significantly influence responses but cannot override higher-level constraints.
50%UnverifiedDefenses against prompt injection (classifiers, instruction hierarchies, source labels) are without exception probabilistic; the model may be bypassed by novel phrasing, and classifiers may misjudge legitimate input.
50%UnverifiedThe fundamental logic of defending against indirect injection is the separation of data and instructions: retrieved content should be confined to a read-only data zone, and hierarchical isolation can be established by placing external content in the tool role rather than the user role.
50%