Prompt Injection
An attack technique targeting large language models in which adversarial instructions are embedded within external content read by the model, such as web pages, documents, or tool return values. This hijacks the model's behavioral logic without requiring direct interaction with the model.
Core Facts
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
Prompt injection attacks are a serious threat facing local AI Agents, as maliciously crafted content in files processed by the Agent may induce it to execute dangerous operations.
OWASP has listed prompt injection as the top security risk for large language model applications.
Timeline (last 90 days)
In a multi-agent environment, untrusted content read by one agent may be passed downstream to another agent as a conclusion or recommendation, and the downstream agent often treats the upstream output as trusted input by default, causing malicious instructions to gain execution permission along the delegation chain.
Prompt injection attackers can probe the target repeatedly with no built-in time constraint, while defenders must take into account the latency of legitimate prompts, creating a structural asymmetry.
Defenses against prompt injection (classifiers, instruction hierarchies, source labels) are without exception probabilistic; the model may be bypassed by novel phrasing, and classifiers may misjudge legitimate input.
LLM hardening refers to reducing the risk of prompt injection, jailbreaking, or data leakage in large language models through means such as system prompt constraints, input filtering, output review, and least privilege.
A single compromised MCP server can execute prompt injection, credential exfiltration, delivery of phishing toolkit fingerprints, PII leakage, token theft via OAuth, and SSRF attacks.
Architects need to defend against prompt injection at multiple levels: explicitly refusing unauthorized instructions in the system prompt, sandboxing tool-returned content, limiting the scope of data agents can access, mandating human approval for high-risk operations, and conducting regular red-team exercises.
The ability to make a model 'ignore all previous instructions' shares the same origin as the prompt injection problem in the security field; attackers exploit the model's tendency to obey instructions that appear later by embedding malicious instructions in web page content or documents to hijack the model's behavior.
In the text fallback scenario, an attacker does not need to control the converter; merely getting a file to fall into the fallback path allows delivery of a carefully crafted instruction string into the LLM context, making that endpoint an open injection payload distributor.
The fallback converter is a security boundary the developer was unaware of: auto-detecting the type without an allowlist turns any unparseable file into text and sends it straight into the LLM context window, constituting a prompt injection delivery system.
The deployment of legal AI must be equipped with protections against prompt injection.
40 more timeline events
All Facts (20)
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
90%VerifiedPrompt injection attacks are a serious threat facing local AI Agents, as maliciously crafted content in files processed by the Agent may induce it to execute dangerous operations.
90%VerifiedOWASP has listed prompt injection as the top security risk for large language model applications.
90%VerifiedPrompt injection is divided into two categories—direct injection and indirect injection—where indirect injection refers to attackers concealing malicious instructions in external content that the AI may read, such as webpage body text, code comments, README files, and image OCR text.
85%VerifiedPrompt Injection is a security attack method targeting AI applications, analogous to SQL injection in traditional web development.
85%VerifiedPrompt engineering refers to the practice of carefully designing input text to guide AI models toward producing higher-quality outputs that better meet expectations.
80%VerifiedIndirect prompt injection attacks plant hidden instructions within normal business data to induce Agents into executing unauthorized operations, representing a security risk at the semantic level rather than the code level.
80%VerifiedIn a Prompt Injection attack, an attacker can craft a special input to induce the Agent to perform unintended operations.
80%VerifiedOWASP已将Prompt注入列为LLM应用十大安全风险之首
80%VerifiedAI safety emphasizes defending against external threats and preventing system misuse, including prompt injection defense, jailbreak protection, data leakage prevention, and abuse detection.
80%VerifiedIf the underlying data is messy, AI models will amplify those errors faster and at greater scale—meaning "garbage in, garbage out" is further magnified in the AI era.
75%VerifiedPrompt injection has become one of the most closely watched attack surfaces in the field of agent security
75%VerifiedThe concept of Prompt Injection was first systematically proposed by security researchers including Simon Willison in 2022.
75%VerifiedInternal agents that are maliciously prompt-injected or misled may execute dangerous operations while holding legitimate permissions.
75%Verified提示注入是一种针对大语言模型的攻击手法，攻击者通过在模型输入数据中嵌入恶意指令劫持模型的正常行为
75%VerifiedJailbreak attacks refer to bypassing a model's safety alignment mechanisms through carefully crafted prompts to make it output harmful content.
75%VerifiedPrompt injection attacks can use carefully crafted user inputs to induce the model to ignore the constraints of the system prompt.
70%VerifiedThere is currently no perfect defense against prompt injection in the industry; the primary approach relies on multi-layered protection strategies, including input filtering, output detection, and the model's own instruction-following capability.
70%VerifiedMCP tool annotations (such as read-only, destructive, idempotent, and open-world) are hints rather than guarantees, and applications should treat them as untrusted information unless the server itself is trusted.
65%VerifiedBecause large models themselves cannot strictly distinguish between instructions and data, prompt injection is fundamentally very difficult to completely eliminate.
65%