Prompt Injection
Prompt injection is an AI security attack technique that exploits the architectural limitation of large language models in distinguishing between data and instructions. By crafting specific inputs, attackers can bypass a model's safety constraints and redirect its behavior, drawing an analogy to traditional SQL injection attacks in conventional software systems.
Core Facts
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
Prompt injection attacks are a serious threat facing local AI Agents, as maliciously crafted content in files processed by the Agent may induce it to execute dangerous operations.
Prompt Injection is a security attack method targeting AI applications, analogous to SQL injection in traditional web development.
Timeline (last 90 days)
Gemini CLI 会合并工作区的 .gemini/settings.json 配置，若允许其设置 defaultApprovalMode 为 yolo，则克隆恶意仓库后可自动执行 shell 命令，形成提示注入加自动批准的严重安全风险。
Retrieved external content (emails, attachments, documents) should be treated as untrusted input, and adversarial documents should be included in evaluation sets.
LLMs have no hard isolation mechanism between data and instructions and must rely on semantic understanding to distinguish the two, which is their vulnerability.
Prompt injection was first publicly demonstrated by Riley Goodside in 2022 and involves disguising instructions as ordinary input content to a model in order to bypass the system prompt.
Hardcoding validation logic on the server side rather than relying on system prompts to constrain model behavior is a fundamental engineering practice for defending against prompt injection attacks
Prompt Injection refers to malicious text being embedded in content that the model will read, in an attempt to override or tamper with the original instructions.
There is currently no fundamental technical solution in the industry for prompt injection, and the main mitigation measures include the principle of least privilege, input sandboxing and filtering, and mandatory human confirmation before sensitive operations
MCP's human-in-the-loop authorization mechanism is consistent with the defense logic against prompt injection attacks: even if an attacker induces an agent to issue an unauthorized request, it cannot bypass the manual checkpoint of user consent.
Agents lacking isolation face indirect prompt injection attacks when scraping local HTML documents or websites containing maliciously hidden prompts, as language models cannot reliably distinguish legitimate commands from hidden payloads embedded in documents.
PinSay treats the text content of comments as untrusted data to reduce the risk of prompt injection attacks
40 more timeline events
All Facts (20)
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
90%VerifiedPrompt injection attacks are a serious threat facing local AI Agents, as maliciously crafted content in files processed by the Agent may induce it to execute dangerous operations.
90%VerifiedPrompt Injection is a security attack method targeting AI applications, analogous to SQL injection in traditional web development.
85%VerifiedCompared to traditional scripts, AI Agents are unpredictable and carry inherent risks such as hallucinations, context misinterpretation, and susceptibility to prompt injection attacks.
80%VerifiedIndirect prompt injection attacks plant hidden instructions within normal business data to induce Agents into executing unauthorized operations, representing a security risk at the semantic level rather than the code level.
80%VerifiedPrompt injection attacks induce AI agents to execute malicious operations by embedding text disguised as system instructions in code comments, documentation, web content, or even file names.
80%VerifiedIn a Prompt Injection attack, an attacker can craft a special input to induce the Agent to perform unintended operations.
80%VerifiedOWASP已将Prompt注入列为LLM应用十大安全风险之首
80%VerifiedAI systems have introduced entirely new dimensions of security risk, including erroneous decisions caused by model hallucinations, prompt injection attacks, training data poisoning, and AI agent privilege abuse.
80%VerifiedPrompt injection has become one of the most closely watched attack surfaces in the field of agent security
75%Verified提示注入是一种针对大语言模型的攻击手法，攻击者通过在模型输入数据中嵌入恶意指令劫持模型的正常行为
75%VerifiedPrompt injection is essentially an architectural flaw in which large language models cannot distinguish data from instructions, analogous to traditional SQL injection in which the database engine confuses data with SQL instructions.
75%VerifiedPrivilege level and data access scope determine the severity of a vulnerability.
70%VerifiedIndirect prompt injection is an attack vector in which an attacker embeds malicious instructions in web pages, documents, or emails that an agent may read, inducing the agent to perform operations beyond the scope of the original task.
70%VerifiedIndirect prompt injection refers to attack instructions hidden in external web pages, documents, or API responses accessed by an agent, where the malicious instructions are activated when the agent retrieves and injects them into the context.
70%VerifiedLarge language models are architecturally incapable of strictly distinguishing between system instructions and user data, which is a fundamental flaw at the architectural level.
70%VerifiedPrompt injection attacks can use carefully crafted user inputs to induce the model to ignore the constraints of the system prompt.
70%VerifiedThe prompt injection security threat category was first systematically proposed by security researcher Simon Willison in 2022.
70%VerifiedAI代理操控计算机面临的安全风险包括提示注入攻击、权限升级和不可逆操作等问题
70%VerifiedIndirect injection refers to attackers hiding malicious instructions in external content that the model will read, such as webpage body text, email body text, or document content.
65%