Data Poisoning
Data poisoning is an AI security attack technique in which malicious content is injected into training data or RAG retrieval sources to manipulate the outputs of AI models. It operates at two levels: training data poisoning, which corrupts the model during the learning phase, and RAG retrieval poisoning, which corrupts the external knowledge sources used at inference time.
Timeline (last 90 days)
Once ECDSA signatures fall, attackers can forge valid cryptographic credentials for malicious data sources, inject backdoors and poison fine-tuning data, making model outputs untrustworthy.
A malicious actor can inject two seemingly harmless independent corpora at different stages of the pretraining pipeline, which combine transitively into an exploit vector during user-led fine-tuning, without injecting an obvious trojan.
Prompt injection, jailbreaking, and data poisoning differ significantly from traditional software vulnerabilities (such as buffer overflows and SQL injection) in their causes and exploitability: they often have no stable patch boundaries, the same prompt behaves differently across model versions, and they are difficult to fully describe using the traditional CVE numbering system.
Data poisoning occurs during the training or fine-tuning stage, where attackers mix malicious samples into the dataset, causing the model to produce erroneous behavior under specific trigger conditions.
AI capabilities are being introduced into areas such as grid load forecasting, water anomaly detection, traffic scheduling, and medical diagnosis, improving efficiency while also introducing new attack surfaces—for example, data poisoning can contaminate scheduling models, and prompt injection can bypass the safety policies of operations assistants.
Research indicates that an attacker needs to modify as little as 0.01% of training data to potentially implant backdoor behaviors in a model that are undetectable by standard evaluations.
知识投毒攻击类似于对抗性机器学习中的数据投毒，但发生在推理层面而非训练层面，因此更加隐蔽且难以检测
All Facts (7)
A malicious actor can inject two seemingly harmless independent corpora at different stages of the pretraining pipeline, which combine transitively into an exploit vector during user-led fine-tuning, without injecting an obvious trojan.
50%UnverifiedAI capabilities are being introduced into areas such as grid load forecasting, water anomaly detection, traffic scheduling, and medical diagnosis, improving efficiency while also introducing new attack surfaces—for example, data poisoning can contaminate scheduling models, and prompt injection can bypass the safety policies of operations assistants.
50%UnverifiedData poisoning occurs during the training or fine-tuning stage, where attackers mix malicious samples into the dataset, causing the model to produce erroneous behavior under specific trigger conditions.
50%UnverifiedPrompt injection, jailbreaking, and data poisoning differ significantly from traditional software vulnerabilities (such as buffer overflows and SQL injection) in their causes and exploitability: they often have no stable patch boundaries, the same prompt behaves differently across model versions, and they are difficult to fully describe using the traditional CVE numbering system.
50%UnverifiedResearch indicates that an attacker needs to modify as little as 0.01% of training data to potentially implant backdoor behaviors in a model that are undetectable by standard evaluations.
50%Unverified知识投毒攻击类似于对抗性机器学习中的数据投毒，但发生在推理层面而非训练层面，因此更加隐蔽且难以检测
50%UnverifiedOnce ECDSA signatures fall, attackers can forge valid cryptographic credentials for malicious data sources, inject backdoors and poison fine-tuning data, making model outputs untrustworthy.
50%