CTF Red Teaming Reveals: Non-Cybersecurity Models Are Better at Finding Sandbox Vulnerabilities

SPACE sandbox CTF red team experiments: non-cybersecurity models are better at finding escape vulnerabilities; DNS spoofing vulnerability fixed.
The SPACE open-source sandbox platform found through CTF red team experiments that non-cybersecurity models outperform security-specialized models at finding sandbox escape vulnerabilities, possibly because the latter are over-aligned and tend to be conservative. CTF prompts must explicitly require breaking host-guest isolation; otherwise, models will not attempt escape, and prompt engineering is easily overlooked. The SPACE sandbox is robust against isolation escapes, and the team discovered and fixed a DNS spoofing vulnerability that bypasses network restrictions.
Red teaming is an important means of evaluating the security of AI systems, but how to design effective test prompts and what kind of model is better suited to play the "attacker" role have long been subjects of industry debate. Recently, an open-source sandbox platform called SPACE, through a series of CTF (Capture The Flag) red team experiments, delivered several counterintuitive findings: non-cybersecurity-specialized models actually perform better than cybersecurity models at finding sandbox escape vulnerabilities.

CTF red teaming requires explicit prompts to "break isolation"
The SPACE team found in red teaming that if the CTF challenge prompt does not explicitly require the model to break host-guest isolation, the model often will not proactively attempt escape behavior. This means test designers need to explicitly instruct "break isolation" in the prompt to effectively trigger the model's aggressive exploration of sandbox boundaries.
This finding has a direct impact on security evaluation methodology: vague CTF tasks may underestimate a model's true attack capability. The prompt engineering of red teaming itself is an easily overlooked link in the security evaluation chain.
CTF (Capture The Flag) is a form of attack-defense exercise originating from cybersecurity competitions, in which participants must exploit vulnerabilities in a controlled environment to obtain a hidden "flag" string as proof of a successful attack. In AI red teaming, CTF has been adapted into a standardized task for evaluating a model's attack capability: testers set up a sandbox environment containing target vulnerabilities and require the model to autonomously discover and exploit vulnerabilities to accomplish a specified goal. Unlike traditional penetration testing, CTF red teaming emphasizes end-to-end autonomy—the model must plan its own attack path, write exploit code, and execute it. This format can more realistically reflect the threat potential of AI agents in open environments, but it also places higher demands on prompt design: if the task description is not clear enough, the model may interpret "obtain the flag" as a legitimate operation rather than a boundary breach, thereby underestimating its actual attack capability.
Non-cybersecurity models are surprisingly better at red teaming
The most surprising conclusion came from model comparison experiments. The SPACE team compared the performance of general-purpose models and cybersecurity-specialized models (e.g., GPT 5.6 Sol vs. GPT 5.6 Sol Cyber) on red team tasks, and found that non-cybersecurity models were actually superior at discovering vulnerabilities.
One possible explanation is that cybersecurity-specialized models may be over-aligned or fine-tuned, making them more conservative in exploratory attack behavior; whereas general-purpose models retain broader reasoning paths and are more creative when searching for unintended vulnerabilities. This finding challenges the intuitive assumption that "specialized security models are better suited for red teaming" and offers security teams a new dimension to consider when selecting red team agents.
Over-alignment refers to a model being excessively reinforced with safety constraints during training, causing it to exhibit overly conservative behavior even in legitimate exploratory scenarios. In red teaming, cybersecurity-specialized models are typically fine-tuned on large amounts of security corpora and trained on refusal behavior, making them more sensitive to "aggressive" instructions and potentially misclassifying exploratory vulnerability discovery as malicious requests, refusing to execute or adopting conservative strategies. In contrast, general-purpose models retain broader reasoning paths and fewer domain-specific constraints, making them more creative in finding unintended vulnerabilities. This phenomenon is related to the concept of the "alignment tax": safety alignment may come at the cost of the model's capability on specific tasks. Therefore, the choice of red team agent requires weighing safety constraints against aggressive exploration capability, rather than simply trusting specialized security models.
SPACE sandbox is robust against isolation escapes
In terms of test results, the SPACE platform showed good robustness against sandbox isolation escape attacks. The team explicitly stated that its sandbox platform remained stable in the face of isolation escape exploits and was not breached.
Sandbox isolation is the core security boundary when AI agents execute untrusted code. SPACE's test results provide some degree of empirical support for its isolation mechanism, but the team also emphasized that red teaming is an ongoing process, and robustness in a single round of testing does not mean permanent security.
DNS spoofing bypassing network restrictions: discovered and fixed
Red teaming was not without its gains. The team disclosed that they discovered an attack path that bypasses network restrictions through DNS spoofing, and have completed a fix for the issue. This vulnerability type is worth vigilance for all agent sandbox platforms: if network policies rely only on domain-layer controls, spoofing at the DNS layer may become a means of bypass.
The SPACE team chose to publicly disclose this finding rather than silently patching it, reflecting its commitment to transparent security disclosure.
DNS spoofing is an attack technique that redirects target traffic to an attacker-controlled address by tampering with domain name resolution results. In agent sandbox scenarios, network restrictions are often implemented through domain whitelists or DNS filtering, allowing agents to access only specific domains. If a sandbox relies only on domain-layer controls and lacks integrity verification of DNS responses, an attacker can forge DNS responses to resolve a legitimate domain to a malicious IP, thereby bypassing network policies. Defending against such attacks requires introducing DNSSEC validation, DNS over HTTPS (DoH), or directly IP-based access control policies at the sandbox network layer, ensuring that network restrictions do not fail due to spoofing of underlying protocols. The SPACE team's discovery and fix of this vulnerability serves as a reminder that agent sandbox platforms need to extend defense in depth to foundational protocol layers such as DNS.
The security philosophy of open source and open sharing
The SPACE platform itself is open source, and the team stated that it hopes to continue publicly sharing research conclusions to serve the agent sandbox and security community. Against the backdrop of the rapid proliferation of AI agents, sandbox security directly affects the trustworthy deployment of agent systems. Opening up red team findings and sharing attack surface information helps the entire ecosystem converge more quickly on more secure implementations.
For teams building or using AI agent sandboxes, these lessons from SPACE provide directly applicable practical guidance: red team prompts must be explicit, model selection need not blindly trust specialized security models, and network-layer defenses need to cover underlying protocols such as DNS.
One-line Summary
SPACE sandbox CTF red team experiments found that non-cybersecurity models are better at finding escape vulnerabilities, and the DNS spoofing vulnerability has been fixed.
Paragraph Summary
The SPACE open-source sandbox platform reached counterintuitive conclusions through a series of CTF red team experiments: non-cybersecurity-specialized models are actually superior to cybersecurity models at finding sandbox escape vulnerabilities, possibly because the latter tend to be conservative due to over-alignment. The experiments also found that CTF prompts need to explicitly require "breaking host-guest isolation"; otherwise, models will not proactively attempt escape, and prompt engineering is an easily overlooked part of security evaluation. The SPACE sandbox showed robustness against isolation escape attacks, but through red teaming the team discovered and fixed a vulnerability in which DNS spoofing bypasses network restrictions. The platform adheres to open source and transparent disclosure, providing practical guidance for AI agent sandbox security: red team prompts must be explicit, model selection need not blindly trust specialized security models, and network defenses need to cover underlying protocols such as DNS.
Related articles

Inside Honey Trap Training: Emotional Dissociation and Anchoring Techniques
How are spies trained for honey trap missions? A trainee reveals emotional dissociation, perfume and keyword anchoring, and target photo binding—psychological techniques that condition reflexes for intelligence work.

Frontier AI Risk Management: Full-Pipeline Practices from Training to Deployment
OpenAI outlines a frontier AI risk management framework: from training-stage model alignment to deployment-stage sandboxing and blocking safeguards, plus post-deployment monitoring—a layered risk control system across the model lifecycle.

DaisyRose-27B: 1,200-Task RL Boosts Qwen3.8 from 5.6% to 12% on Terminal-Bench 4.1
DaisyRose-27B uses RL training on 1,200 tasks to boost Qwen3.8's Terminal-Bench 4.1 score from 5.6% to 12%, showing lightweight RL can more than double terminal task performance.