UK AISI Evaluates GPT-5.5 Cybersecurity Capabilities: On Par with Claude Mythos but Already Publicly Available

Overview

The UK AI Safety Institute (AISI) recently published its evaluation report on OpenAI's latest model, GPT-5.5, focusing on its cybersecurity capabilities. The institute had previously conducted a similar assessment of Anthropic's Claude Mythos. The results show that GPT-5.5's ability to discover security vulnerabilities is on par with Claude Mythos, but with one critical difference—GPT-5.5 is already publicly available.

Evaluation Context: Why AI Model Cybersecurity Capabilities Matter

As large language models rapidly advance in capability, their potential applications and risks in the cybersecurity domain are receiving increasing attention. Whether AI models can autonomously discover security vulnerabilities in software is a question with implications on two levels:

• Defense: AI efficiently discovering vulnerabilities allows security teams to leverage it for hardening systems
• Offense: The same capabilities could be exploited by malicious actors to find attack entry points

From a technical mechanism perspective, large language models' ability to discover security vulnerabilities primarily relies on their training across massive code repositories and security research literature. Models can identify common vulnerability patterns (such as buffer overflows, SQL injection, cross-site scripting, etc.) and apply these patterns to new code audit scenarios. More advanced models can also perform multi-step reasoning, simulating attack chains—combining multiple low-severity vulnerabilities into high-risk attack paths. This type of capability previously required experienced penetration testing experts days or even weeks to accomplish, while AI models can complete initial scans in minutes.

For this reason, the UK AISI has made cybersecurity capability evaluation of AI models one of its core tasks. Established in November 2023, the UK AI Safety Institute is the world's first institution set up by a national government specifically to conduct safety evaluations of frontier AI models. It grew out of commitments made at the 2023 Bletchley Park AI Safety Summit hosted by the UK, with the mission of conducting independent technical assessments of the most advanced AI systems, covering multiple risk dimensions including biosecurity, cybersecurity, and autonomy. AISI has established pre-release model evaluation partnerships with major AI labs including OpenAI, Anthropic, and Google DeepMind, enabling it to gain testing access before or shortly after public deployment. The earlier evaluation of Claude Mythos established a methodological framework, and this GPT-5.5 evaluation continues along the same path.

AISI's Evaluation Methodology

AISI's cybersecurity evaluations typically employ a multi-layered testing approach: standardized CTF (Capture The Flag) challenges, reproduction tests of real-world CVE (Common Vulnerabilities and Exposures) vulnerabilities, and autonomous penetration testing of simulated systems in controlled environments. The evaluation examines not only whether the model can identify known vulnerability types, but also tests its potential to discover zero-day vulnerabilities (vulnerabilities that have not yet been publicly disclosed or patched), as well as the degree of autonomy in completing full attack chains without human guidance. This tiered evaluation framework helps distinguish the boundary between "assistive" capabilities (the model as a tool for human security researchers) and "transformative" capabilities (the model independently completing end-to-end attack or defense tasks).

Evaluation Results: GPT-5.5 On Par with Claude Mythos

Vulnerability Discovery Capability Comparison

AISI's evaluation focused on the models' ability to discover security vulnerabilities. According to the report, GPT-5.5's performance on this dimension is at the same level as Anthropic's Claude Mythos. This suggests that current top-tier AI models are trending toward capability convergence in cybersecurity tasks—the gap between flagship models from different vendors is narrowing on specific security tasks.

The technical root cause of this capability convergence phenomenon lies in the fact that major labs use highly overlapping training data sources (publicly available internet data, academic papers, open-source code, etc.), employ the same foundational architecture (Transformer), and use increasingly similar training methodologies (pre-training + RLHF/RLAIF and other reinforcement learning from human feedback methods). Once model scale reaches a certain threshold, performance differences on standardized benchmarks often narrow to within statistical error margins. This trend implies that any single vendor's "competitive advantage" in cybersecurity capabilities may be only temporary, as the overall industry capability level is rising in sync.

Key Difference: Security Challenges from Public Availability

Although the two models are comparable in capability, there is one important real-world difference: GPT-5.5 is currently publicly available, while Claude Mythos had not been fully released at the time of evaluation. This means GPT-5.5's cybersecurity capabilities are already "in the wild," accessible to anyone through APIs or product interfaces.

A model's public availability is a core variable in AI safety discussions. When a model runs only in controlled environments, developers can reduce misuse risk through usage policies, rate limiting, and output filtering. But once a model is widely deployed through APIs or consumer-facing products, the effectiveness of these controls drops significantly. Attackers can bypass safety guardrails through techniques like prompt injection and jailbreaking, or simply use model outputs as supplementary reference for attack research. More concerning is that even when a model has safety restrictions, some of the information it generates—such as explanations of vulnerability principles and analysis of attack vectors—may still provide valuable leads to attackers with some technical background.

This difference poses more urgent demands on AI safety governance—when models capable of vulnerability discovery are already widely available, defenders need to adapt to this new reality more quickly. Security teams can no longer assume that attackers lack automated vulnerability discovery tools, and should incorporate AI-assisted attacks into the baseline assumptions of their threat models.

Broader Industry Implications

AI Safety Evaluation Becoming Institutionalized

The UK AISI's sequential cybersecurity evaluations of multiple frontier models signals that AI safety evaluation is becoming institutionalized and routine. This type of independent evaluation led by government agencies provides relatively objective reference points for the public and policymakers, while also setting external benchmarks for AI vendors' security practices.

Notably, AISI's evaluation model is influencing the global AI governance landscape. The US AI Safety Institute (under NIST), the EU AI Office, and similar agencies in other countries are all establishing their own evaluation frameworks. The formation of this multilateral evaluation system is expected to drive international coordination on AI safety evaluation standards, avoiding regulatory fragmentation. For AI developers, evaluation results from multiple independent agencies also provide opportunities for cross-validation, enhancing the credibility of evaluation conclusions.

The Double-Edged Sword Effect in Cybersecurity

Interestingly, AI's growing capabilities in cybersecurity represent a classic double-edged sword. The same model can be used by security researchers for vulnerability scanning and code auditing, while also potentially being used by attackers for automated vulnerability mining. As model capabilities continue to improve, how to balance promoting security research with preventing misuse will become a core challenge for the industry and regulators.

From a historical perspective, this double-edged sword effect is nothing new in security tooling—classic security tools like Metasploit and Nmap similarly serve both legitimate penetration testing and potential malicious use. But the qualitative change AI models bring is their "barrier-lowering" effect: traditional security tools require users to have deep technical backgrounds, while AI models can enable users with lower technical skills to perform complex security analysis tasks through natural language interaction. This "democratization" effect is positive on the defense side (more organizations can conduct security audits), but concerning on the attack side (the technical barrier to attacks is significantly lowered).

OpenAI vs. Anthropic Competitive Landscape

From a technical competition perspective, the parity between GPT-5.5 and Claude Mythos in cybersecurity capabilities once again confirms the current "multi-polar" landscape in AI. The capability gap between the two companies is narrowing, and competitive focus may gradually shift toward deployment speed, ecosystem development, and safety governance.

The differences in security philosophy between the two companies are also noteworthy. Anthropic has long positioned itself as an "AI safety company," emphasizing its Constitutional AI methodology and cautious deployment strategy; OpenAI tends toward faster productization and broader public deployment. This evaluation result happens to illustrate the real-world consequences of these strategic differences: models with comparable capabilities produce different social risk profiles due to different deployment strategies. This also raises a deeper question for the industry—when it comes to the pace of deploying frontier AI capabilities, which path is more conducive to overall safety: "rapid iteration with broad availability" or "careful evaluation with controlled release"?

Conclusion

The UK AISI's cybersecurity evaluation of GPT-5.5 provides an important observation window: frontier AI models' cybersecurity capabilities are rapidly advancing and converging toward homogeneity, and these capabilities have already reached a broad user base through public products. For the security community, this is both an opportunity and a warning—while leveraging AI to enhance defensive capabilities, we must confront the risks of misuse and push for the establishment of corresponding evaluation and governance mechanisms.

Looking ahead, as AI model capabilities continue to climb, the cybersecurity field may enter a new "AI vs. AI" phase: defenders using AI for real-time vulnerability detection and automated patching, while attackers leverage AI for automated vulnerability mining and attack generation. In this technological arms race, the role of independent third-party evaluation bodies (such as AISI) will become increasingly critical—they not only provide capability benchmarks but also offer evidence-based decision support for policymakers.

Key Takeaways

• The UK AI Safety Institute (AISI) has completed its evaluation of OpenAI GPT-5.5's cybersecurity capabilities
• GPT-5.5's ability to discover security vulnerabilities is on par with Anthropic's Claude Mythos
• The key difference is that GPT-5.5 is already publicly available, while Claude Mythos had not been fully released at the time of evaluation
• AI model cybersecurity capability evaluation is becoming institutionalized, with government-led independent assessments increasingly routine
• Frontier AI models' advancing security capabilities are a double-edged sword, requiring balance between promoting security research and preventing misuse
• The trend toward capability convergence means safety governance cannot rely on any single vendor's self-discipline, but requires systematic institutional safeguards