Prompt Injection
A security attack targeting LLM-based applications in which an attacker embeds malicious instructions within external content processed by an agent, causing the agent to disregard its original system prompt and execute unauthorized actions. This attack vector is particularly harmful in multi-agent systems, where a compromised agent can propagate malicious behavior across interconnected components.
Timeline (last 90 days)
RAGGuardrails类实现输入和输出护栏：输入检查拦截Prompt注入（检测'ignore previous instructions'等触发词）和PII泄露（正则匹配SSN、邮箱、信用卡号、电话号码）；输出检查包含幻觉检测（提取答案中的数字，检查是否在上下文中，长度超过2位的数字若不在上下文则标记为未验证声明）和PII检测，并在PII出现时用[REDACTED]替换。
Agents may execute destructive operations far beyond what was intended due to hallucination or prompt injection attacks.
即使prompt注入成功改变了模型行为，若容器层面锁死了网络、文件系统和API访问，模型也无法执行危险操作
All Facts (3)
RAGGuardrails类实现输入和输出护栏：输入检查拦截Prompt注入（检测'ignore previous instructions'等触发词）和PII泄露（正则匹配SSN、邮箱、信用卡号、电话号码）；输出检查包含幻觉检测（提取答案中的数字，检查是否在上下文中，长度超过2位的数字若不在上下文则标记为未验证声明）和PII检测，并在PII出现时用[REDACTED]替换。
50%UnverifiedAgents may execute destructive operations far beyond what was intended due to hallucination or prompt injection attacks.
50%Unverified即使prompt注入成功改变了模型行为，若容器层面锁死了网络、文件系统和API访问，模型也无法执行危险操作
50%