Cybench
A cybersecurity AI capability benchmark jointly built by academics and security researchers. It covers CTF challenge problems, vulnerability exploitation, reverse engineering, cryptographic analysis, and other security tasks, aimed at systematically evaluating AI models in real-world offensive and defensive scenarios.
Timeline (last 90 days)
Mistral Large 4 在网络安全基准 Cybench（40 个 CTF 挑战）上得分 93%，在漏洞复现与修补测试中得分 82%，在 B3 AI 安全基准上得分 93.3%。
OrcaCyber Zero 1.5 achieved a score of 100% on the Cybench benchmark.
Cybench is a benchmark designed to evaluate cybersecurity capabilities, covering multiple categories of security tasks including CTF challenges, exploit development, reverse engineering, and cryptographic analysis.
2024年以来多个研究团队（如NYU的Cybench、UIUC的研究）证明大模型在Web和密码学类CTF题目上能达到初中级选手水平，但在Pwn和逆向工程题目上仍存在显著差距
All Facts (4)
Mistral Large 4 在网络安全基准 Cybench（40 个 CTF 挑战）上得分 93%，在漏洞复现与修补测试中得分 82%，在 B3 AI 安全基准上得分 93.3%。
50%UnverifiedOrcaCyber Zero 1.5 achieved a score of 100% on the Cybench benchmark.
50%UnverifiedCybench is a benchmark designed to evaluate cybersecurity capabilities, covering multiple categories of security tasks including CTF challenges, exploit development, reverse engineering, and cryptographic analysis.
50%Unverified2024年以来多个研究团队（如NYU的Cybench、UIUC的研究）证明大模型在Web和密码学类CTF题目上能达到初中级选手水平，但在Pwn和逆向工程题目上仍存在显著差距
50%