Wolf Defender Retraining in Practice: How Hard Negatives Cut False Positive Rate from 33% to 3%

In the field of LLM security, prompt injection detection is an essential line of defense. Prompt injection is an attack technique targeting large language model applications, where attackers craft input text designed to override, tamper with, or bypass a system's preset instructions (system prompt), causing the model to perform unintended behaviors. This is analogous to SQL injection in traditional web security—exploiting the blurred boundary between input and instructions to achieve malicious manipulation. As LLMs are widely deployed in customer service, code generation, data analysis, and more, OWASP has ranked prompt injection as the number one security risk for LLM applications.

However, an overly aggressive classifier that frequently flags normal inputs as attacks can become a stumbling block for business operations. Recently, the Patronus Studio team retrained their prompt injection classifier, Wolf Defender, from scratch. The core motivation wasn't insufficient attack detection capability—it was too many false positives. In their words, the model kept "crying wolf."

False Positives Are the Real Pain Point: The Core Problem with the Old Model

The team explicitly stated that the old model already performed well at detecting prompt injections. The real pain point was false positives—especially on short benign inputs, security-related text, code snippets, and ordinary conversations, where the model was far too aggressive.

False positives are a core challenge facing all security detection systems, whether network intrusion detection systems (IDS), spam filters, or AI security classifiers. In statistics, there is an inherent tension between the false positive rate and the true positive rate, a relationship typically visualized with ROC curves. In production environments, the cost of false positives is often severely underestimated: every false block means degraded user experience and disrupted business workflows. The well-known "alert fatigue" phenomenon in the security industry—where excessive false alerts cause operators to gradually ignore all alerts, ultimately missing real threats—is an extreme manifestation of false positive harm.

A highly representative example: a user simply typed "Who are you?" The old Wolf Defender Small classified it as a prompt injection attack with roughly 94% confidence. False positives like this clearly interfere severely with normal user interactions.

This phenomenon reveals a universal dilemma in security classifier design: a model scoring near-perfect on clean validation sets doesn't mean it will perform well on real-world traffic. When legitimate requests are frequently blocked, even the highest attack detection rate loses its practical value.

Why Do Short Texts Trigger More False Positives?

Short inputs often lack sufficient contextual information, making it difficult for models to distinguish reasonable requests like "asking about identity" from real attacks like "ignore previous instructions." Additionally, security documentation, technical explanations about prompt injection, and system policy language naturally contain many "suspicious" words and structures that mislead overly sensitive classifiers. From an information-theoretic perspective, short texts have lower entropy, giving the model very limited discriminative features to work with—any slight bias can tip the classification toward the wrong result.

Wolf Defender v2 Training Strategy: Targeting Hard Negatives

To address the false positive epidemic, the team made significant adjustments to the training pipeline. Both Wolf Defender and Wolf Defender Small were retrained from scratch using a fresh mmBERT checkpoint, with a focus on hard negatives.

mmBERT (multilingual miniature BERT) is a compact multilingual variant of the BERT model family. BERT was introduced by Google in 2018 as a milestone in pretrained language models, with its core innovation being bidirectional attention—learning semantic representations by simultaneously considering context on both sides of a word. For classification tasks, the BERT architecture is more efficient than generative large models (such as the GPT series) because it doesn't need to autoregressively generate output token by token; it simply encodes the input and outputs a classification through a classification head. Choosing to retrain from an mmBERT checkpoint rather than fine-tuning the existing model meant the team wanted to avoid inheriting biases the old model had learned (such as oversensitivity to short text), starting from a cleaner foundation to rebuild classification capabilities.

Hard negatives are inputs that "look suspicious but actually have no manipulative intent." Hard negative mining is a classic technique in contrastive learning and metric learning: simple negatives contribute little to model learning, while samples near the decision boundary are what truly drive refinement of that boundary. For example, a technical blog post discussing "how to defend against prompt injection" has high lexical overlap with actual prompt injection attacks at the word level, but the intent is completely different. By introducing large quantities of such hard negatives, the model is forced to learn deeper semantic features rather than surface-level keyword patterns.

Specific hard negatives included:

• Short conversations, emails, technical documentation
• Explanatory text about prompt injection itself
• Benign policy and system language
• Code and configuration snippets

Beyond these, the team also added more counterfactual samples, multilingual samples, adversarial obfuscation, and long-context injections placed at various positions within documents. This targeted data construction directly addressed the scenarios where the old model was most prone to failure.

Key Training Techniques Explained

In terms of training methodology, v2 combined short samples (256 tokens) with full-length windows (2,048 tokens) for mixed training, along with several advanced techniques:

• Supervised Contrastive Regularization: This method originates from the supervised contrastive learning framework (SupCon) proposed by Google Research in 2020, which is a supervised extension of self-supervised contrastive learning methods (like SimCLR and MoCo). The core idea is to pull same-class samples closer together in representation space while pushing different-class samples apart. For each anchor sample, all same-class samples form positive pairs and all different-class samples form negative pairs, optimizing the representation distribution through a modified InfoNCE loss function. Adding this as a regularization term to the classification loss effectively prevents the model from overfitting to surface features of the training set, enabling benign and malicious inputs to form clearer cluster structures in high-dimensional space—which is especially critical for security classification tasks with ambiguous boundaries.

• FreeLB Adversarial Training: FreeLB (Free Large-Batch Adversarial Training) was proposed by Microsoft Research in 2019, specifically optimized for NLP tasks. In NLP, since inputs are discrete token sequences, adversarial perturbations are typically applied in the continuous embedding space. FreeLB's innovation lies in performing multi-step PGD attacks on the embedding space within each training step and accumulating gradients from all perturbation steps for a single parameter update—equivalent to adversarial training on a larger virtual batch, but with computational overhead of only a fraction of multi-step PGD. In prompt injection detection scenarios, attackers frequently mutate attack payloads through synonym substitution, character obfuscation (such as zero-width characters), multilingual mixing, and other techniques. FreeLB training enables the model to maintain stable classification results against these small perturbations in embedding space.

• Smooth-Max Aggregation: Since Transformer models like BERT have fixed context window limits, processing ultra-long documents requires splitting them into multiple segments for separate inference, then aggregating segment predictions into a final judgment. Simple Max aggregation (taking the highest attack confidence across all segments) easily leads to false positives—if any single segment looks slightly suspicious, the entire document gets flagged as an attack. Average aggregation, on the other hand, may dilute genuine attack signals. Smooth-Max uses a temperature parameter to control the "sharpness" of aggregation: higher temperature approaches averaging, lower temperature approaches max, thereby avoiding whole-document false positives triggered by local segments while effectively capturing real injection attacks hidden somewhere in a long document.

The goal of this combined approach is clear: train the model to recognize semantic intent rather than keyword patterns.

Results: False Positive Rate Drops from 33% to 3%

The retraining results are strikingly visible on benign benchmarks. Here's the v1 vs. v2 comparison:

Model	Benchmark	v1	v2
Wolf Defender	Hard Benign Specificity	81.57%	96.23%
Wolf Defender	Real-World Benign Specificity	66.85%	96.63%
Wolf Defender Small	Hard Benign Specificity	82.12%	96.67%
Wolf Defender Small	Real-World Benign Specificity	73.60%	94.38%

Specificity here, also known as the True Negative Rate (TNR), is the core metric for measuring the ability to "correctly pass through normal inputs." It's calculated as: Specificity = True Negatives / (True Negatives + False Positives). "Hard Benign Specificity" measures the model's correct classification rate on inputs whose surface features resemble attacks but are actually benign, while "Real-World Benign Specificity" reflects the model's pass-through rate for normal requests in real production traffic.

The most striking improvement is in Real-World Benign Specificity: the main model jumped from 66.85% to 96.63%, and the Small model improved from 73.60% to 94.38%. This means a dramatic reduction in falsely blocked legitimate requests in real traffic—for production systems handling millions of requests daily, the main model's improvement translates to roughly 300,000 fewer unnecessary blocks per day.

Meanwhile, attack detection capability remained essentially unchanged:

Model	Qualifire F1	Jayavibhav F1
Wolf Defender	95.14%	97.84%
Wolf Defender Small	95.21%	97.68%

Back to that classic example: "Who are you?" is now classified by Wolf Defender Small v2 as benign with 98.55% confidence, while a genuine instruction override attack is still detected at 99.99% confidence.

Engineering Trade-offs: Sacrificing Perfect Benchmarks for Real-World Performance

The team candidly acknowledged that this retraining involved explicit trade-offs. On cleaner validation distributions, some previously sky-high scores showed slight decreases.

But they consider this entirely worthwhile: "A security classifier that scores near-perfect on benchmarks but constantly blocks normal traffic isn't very useful. We'd rather sacrifice a bit on easy validation sets in exchange for significantly better performance on real benign inputs."

This judgment reflects mature engineering thinking—benchmark scores aren't the finish line; real-world deployment performance is. Overfitting to idealized test sets often comes at the cost of real-world performance. This phenomenon is a manifestation of what's known in machine learning as "Goodhart's Law": when a metric becomes a target, it ceases to be a good metric. Over-pursuing perfect scores on validation sets can actually cause the model to learn distribution-specific features of the validation set rather than genuine discriminative ability.

Deployment Options: Full Coverage from FP32 to INT4

On the deployment front, the team updated their deployment variants. Both models are available as standard Transformers checkpoints and in ONNX export format, covering:

• FP32 (full precision, 32-bit floating point)
• FP16 (half precision, 16-bit floating point)
• INT8/FP16 mixed precision
• INT8 with INT4 embeddings

Model quantization is a technique that converts neural network weights and activation values from high-precision floating point to lower-precision representations, aiming to reduce model size, lower memory usage, and accelerate inference. Quantization from FP32 to INT8 typically shrinks model size by approximately 4x, with further INT4 quantization achieving up to 8x compression. ONNX (Open Neural Network Exchange) is an open format jointly developed by Microsoft and Facebook that enables seamless model migration across different inference frameworks (such as ONNX Runtime, TensorRT, and OpenVINO), which is essential for cross-platform deployment.

After quantization, the smallest Wolf Defender Small artifact is only 96 MB, which is quite friendly for scenarios requiring security gateway deployment on edge devices or in high-concurrency environments. The 96MB size makes it deployable at the API gateway layer as a pre-filter, performing real-time detection on every incoming request without GPU acceleration—this is significant for cost-sensitive enterprise deployments or CDN scenarios that need security filtering at edge nodes close to users. The model files have been open-sourced on Hugging Face.

Implications for Prompt Injection Defense Practices

This retraining case offers several practical lessons for security classifier design and optimization:

First, false positives are one of a security classifier's true enemies. When attack detection is already good enough, reducing false positives often delivers greater real business value. A security system's ultimate goal isn't pursuing perfect detection rates in the lab, but achieving the optimal balance between attack blocking and user experience in real environments.

Second, constructing hard negatives is critical. Only by exposing the model to enough inputs that "look suspicious but are harmless" can it develop the ability to distinguish intent from keywords. This approach mirrors the growth path of human security analysts—experienced analysts reduce false judgments precisely because they've seen vast numbers of "false alarm" cases.

Third, don't be fooled by impressive benchmark scores. Real traffic distributions are far more complex than validation sets. Strategically sacrificing idealized metrics for real-world performance is often the wiser choice.

Fourth, quantized deployment lowers the barrier to entry. A 96MB model size makes high-frequency invocation and edge deployment feasible—security protection doesn't have to come at the cost of expensive inference.

Finally, the team posed an open question to the community: if you're running a prompt injection classifier on real traffic, which benign inputs still most frequently trigger false positives? This remains a direction worth continued exploration across the field.

Key Takeaways