CVE-Bench
An AI evaluation benchmark constructed from publicly disclosed CVE entries, examining a model's ability to identify, classify, and analyze the exploitation of known vulnerabilities. Some entries cannot be automatically evaluated due to incomplete information or difficulty in reproduction.
Timeline (last 90 days)
OrcaCyber Zero 1.5 achieved a score of 95.8% on the evaluable subset of CVE-Bench.
CVE-Bench is based on publicly disclosed CVE entries and tests a model's ability to identify, classify, and analyze the exploitability of known vulnerabilities.
A considerable number of entries in CVE-Bench cannot be automatically evaluated due to incomplete information and the difficulty of standardizing reproduction environments, so the evaluable subset typically covers only a portion of the total.
The sample distribution represented by the 95.8% figure may deviate from the types of vulnerabilities encountered in real-world work, as entries that are difficult to evaluate automatically tend to be high-severity vulnerabilities with complex logic and longer exploit chains.
All Facts (4)
OrcaCyber Zero 1.5 achieved a score of 95.8% on the evaluable subset of CVE-Bench.
50%UnverifiedCVE-Bench is based on publicly disclosed CVE entries and tests a model's ability to identify, classify, and analyze the exploitability of known vulnerabilities.
50%UnverifiedA considerable number of entries in CVE-Bench cannot be automatically evaluated due to incomplete information and the difficulty of standardizing reproduction environments, so the evaluable subset typically covers only a portion of the total.
50%UnverifiedThe sample distribution represented by the 95.8% figure may deviate from the types of vulnerabilities encountered in real-world work, as entries that are difficult to evaluate automatically tend to be high-severity vulnerabilities with complex logic and longer exploit chains.
50%