Prompt Injection
An attack technique targeting AI systems in which malicious instructions are embedded in text read by an AI model, inducing it to deviate from its original objectives and execute attacker-designed behaviors. It is a central security threat facing agents powered by large language models.
Core Facts
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
Prompt injection attacks induce AI agents to execute malicious operations by embedding text disguised as system instructions in code comments, documentation, web content, or even file names.
Prompt injection has become one of the most closely watched attack surfaces in the field of agent security
Timeline (last 90 days)
Prompt injection refers to an attacker embedding malicious instructions into text that the model will read, causing the model to mistake these instructions for part of a legitimate task and execute them.
OWASP classifies prompt injection into two types: direct injection (user input manipulates the model) and indirect injection (the attack payload is hidden in external content read by the model).
Injection and data probing attacks occur with lower probability than the first four groups in real customer interactions, but their severity level is the highest, so they must be included in testing.
Large language models are fundamentally unable to distinguish between system instructions and user input at a structural level, making prompt injection attacks technically impossible to fully eliminate.
OWASP lists prompt injection as the top risk in the LLM Top 10 (LLM01).
Filter/detection-based defenses (such as scanning for phrases like 'ignore previous instructions') are structurally doomed to fail, because natural language has no syntax that distinguishes instructions from data.
A stateful rule: once a session has read content resembling a secret key, that session's subsequent external outbound communication is shut down, blocking "read key then send" attacks.
When all three elements of the deadly triad appear simultaneously in a single session, one successful injection can read sensitive information and exfiltrate it.
Classifiers are probabilistic; a filter that blocks 95% of attempts leaves attackers a 5% opening, while attackers have unlimited retries.
Currently, neither academia nor industry has a perfect defense against prompt injection; common mitigation measures include separating external content from system instructions, restricting permissions for high-risk tools, and requiring mandatory human confirmation before sensitive operations.
30 more timeline events
All Facts (20)
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
90%VerifiedPrompt injection attacks induce AI agents to execute malicious operations by embedding text disguised as system instructions in code comments, documentation, web content, or even file names.
80%VerifiedPrompt injection has become one of the most closely watched attack surfaces in the field of agent security
75%Verified提示注入是一种针对大语言模型的攻击手法，攻击者通过在模型输入数据中嵌入恶意指令劫持模型的正常行为
75%VerifiedPrompt injection is essentially an architectural flaw in which large language models cannot distinguish data from instructions, analogous to traditional SQL injection in which the database engine confuses data with SQL instructions.
75%VerifiedPrompt injection attacks can use carefully crafted user inputs to induce the model to ignore the constraints of the system prompt.
70%VerifiedBecause large models themselves cannot strictly distinguish between instructions and data, prompt injection is fundamentally very difficult to completely eliminate.
65%VerifiedPrompt injection induces agents to deviate from the original task intent by embedding text disguised as instructions within external content processed by the agent.
65%VerifiedPrompt injection can be divided into two types: direct injection and indirect injection; indirect injection is where an attacker hides malicious instructions in external data sources such as web pages, PDFs, and emails, and when an AI agent processes this data the malicious instructions are executed as legitimate instructions.
65%VerifiedThe principle of indirect prompt injection is that an attacker disguises malicious instructions as normal data, delivers them to the model through a data source trusted by the agent, and induces the model to interpret the data as instructions and execute them.
65%VerifiedLLMs are vulnerable to prompt injection attacks, where malicious web pages or files can embed hidden instructions to induce AI to perform unintended actions.
65%UnverifiedSkills Store的审核机制包括检查是否存在提示词注入风险或数据泄露隐患
75%Partially VerifiedOWASP lists prompt injection as the top risk in the LLM Top 10 (LLM01).
65%UnverifiedThe essence of prompt injection is that attackers hijack the execution logic of large language models through carefully crafted text content.
60%UnverifiedResearchers have found that hidden character techniques can be used to launch prompt injection attacks against large language models, embedding hidden instructions within normal text that the model will parse and execute.
60%UnverifiedPrompt injection refers to an attacker embedding malicious instructions into text that the model will read, causing the model to mistake these instructions for part of a legitimate task and execute them.
50%UnverifiedOWASP classifies prompt injection into two types: direct injection (user input manipulates the model) and indirect injection (the attack payload is hidden in external content read by the model).
50%UnverifiedInjection and data probing attacks occur with lower probability than the first four groups in real customer interactions, but their severity level is the highest, so they must be included in testing.
50%UnverifiedLarge language models are fundamentally unable to distinguish between system instructions and user input at a structural level, making prompt injection attacks technically impossible to fully eliminate.
50%UnverifiedClassifiers are probabilistic; a filter that blocks 95% of attempts leaves attackers a 5% opening, while attackers have unlimited retries.
50%