Cloudflare Open-Sources security-audit-skill: Bringing Verifiable AI Security Auditing to Coding Agents

When AI Coding Assistants Start Doing Security Audits

Cloudflare's recently open-sourced security-audit-skill project has quickly gained traction on GitHub, adding over 500 stars in a single day and surpassing a cumulative total of 25,000. Its purpose is clear: it's a "skill" designed to be called by coding agents, enabling multi-phase security audits that produce independently verified, machine-readable findings.

The project has attracted attention primarily because it tackles two of the thorniest problems in AI-assisted security auditing — result trustworthiness and consumability. Traditionally, asking a large language model to scan code and output "there might be a vulnerability here" produces results that are neither reliable nor easy to process automatically. Cloudflare's approach is to break the audit into multiple phases and introduce an independent verification step to filter out hallucinations and false positives.

Multi-Phase Auditing: Why Break It Into Stages?

The "multi-phase" aspect of the project name is central to its design philosophy. A single LLM call attempting to complete a full security analysis is prone to incomplete coverage, context overflow, and highly variable conclusions. Splitting the audit workflow into distinct phases — such as discovery, analysis, and verification — allows each step to focus on a narrower task scope, improving overall accuracy.

This staged approach aligns with the mainstream direction of agent engineering in recent years: rather than chasing a single "all-knowing" inference, the better path is to decompose complex tasks into composable, reviewable sub-steps. For security audits, which are highly sensitive to both false positives and false negatives, phasing brings an additional benefit: intermediate outputs can be inspected, making the entire process transparent rather than a black box.

Independent Verification: The Key to Fighting AI Hallucinations

"Independently verified" is the most interesting design choice in this project. The biggest trust barrier for AI in code auditing is that models will confidently report vulnerabilities that simply don't exist, or completely miss real issues. By emphasizing that findings must pass independent verification, security-audit-skill ensures that initial judgments aren't treated as final conclusions — they must be confirmed through a separate mechanism.

This "discovery–verification" separation architecture is essentially an engineering-level validation gate placed around the LLM's output. For security teams, this means the reports they receive have higher credibility, reducing the cost of manually reviewing large volumes of false positives.

AI hallucinations are particularly harmful in code security auditing. When generating code analysis conclusions, large language models may — due to training data bias or insufficient contextual understanding — confidently report an SQL injection that can't possibly be triggered given the actual code logic, or flag a buffer operation that already has boundary checks as an overflow risk. When such false positives accumulate, they seriously drain security engineers' attention and create a "cry wolf" effect, causing real vulnerabilities to be overlooked. Common approaches to independent verification include: using a second model or rule engine to cross-check initial findings; automatically generating PoC (proof-of-concept) code and executing it in a sandbox to confirm exploitability; or using static analysis tools to verify the reachability of code paths flagged by the model. The specific mechanism used by security-audit-skill hasn't been fully disclosed, but the "discovery–verification" dual-phase architecture itself is already recognized as one of the best practices for combating LLM false positives.

Machine-Readable Audit Findings

Another key phrase is "machine-readable findings." Outputting audit conclusions in a structured, machine-readable format — rather than as a block of natural language — allows results to be directly consumed by downstream toolchains, such as CI/CD pipelines, vulnerability management platforms, or automated ticketing systems.

In DevSecOps practice, "shift-left security" requires security checks to integrate seamlessly into the development workflow. Structured output is the prerequisite for making this happen. A report that only humans can read is difficult to automate, whereas machine-readable findings can trigger automatic blocking, tiered alerts, or regression validation — truly embedding AI audit capabilities into the engineering pipeline.

DevSecOps (Development, Security, and Operations) is an engineering culture and methodology that integrates security practices throughout the entire software development lifecycle. Its core concept is "shift-left security" — moving security checks from pre-launch audits forward into the development and CI/CD stages, so that security issues are caught and fixed at the lowest possible cost. Machine-readable findings are typically output in standard formats such as SARIF (Static Analysis Results Interchange Format), JSON Schema, or CycloneDX. These formats are natively supported by major platforms like GitHub Advanced Security, Jira, and Snyk, and can directly trigger Pull Request blocking, ticket creation, or vulnerability database writes without manual transcription. By contrast, natural-language audit reports require dedicated personnel to read, evaluate, and manually enter data — a process that simply doesn't scale at the pace of modern high-frequency releases. Structured output has therefore become the minimum requirement for AI security tools to enter enterprise engineering pipelines.

Tech Stack and Positioning

Based on repository information, the project is primarily written in JavaScript and exists as a skill for coding agents rather than a standalone scanner. This suggests it's designed to be embedded within existing agent frameworks or workflows, called by agents at the appropriate moment — rather than being manually run as a traditional SAST tool.

This "skill-ified" packaging reflects a broader trend in the agent ecosystem toward modularity: capabilities are broken into individual callable skills that agents dynamically combine based on the task at hand. By turning security auditing into a standardized skill, Cloudflare is filling a critical gap in its agent ecosystem — adding security as a key piece of the puzzle.

The concept of "skills" or "tools" in agent engineering originates from agent architectures represented by frameworks like LangChain and AutoGPT. In these architectures, the large model itself is responsible only for reasoning and decision-making, while specific external actions — such as executing code, searching documents, or calling APIs — are encapsulated as independent callable units (tools/skills/functions). The model invokes these on demand via function calling or the Action mechanism in the ReAct framework. This design allows capability modules to be reused across different agents, and makes it easy to independently test or replace a single capability without affecting the overall reasoning logic. By packaging security auditing as a skill rather than a standalone tool, Cloudflare ensures it can be plugged into any agent framework that supports a standard tool-use interface. Security auditing capability thus evolves from a "one-off script" into a composable, standardized component in the agent ecosystem.

Implications for Security Engineering

The rapid rise of security-audit-skill reflects a strong industry demand for "trustworthy AI security auditing." The answer it offers isn't a bigger model — it's better engineering architecture: multi-phase design to reduce per-step complexity, independent verification to combat hallucinations, and structured output to ensure automation compatibility.

For teams evaluating whether to bring AI into their security workflows, this project offers at least one valuable reference paradigm. Rather than debating whether the LLM itself is reliable enough, the better question is: how do you use process design to make its outputs controllable, verifiable, and integrable? As more organizations incorporate AI coding assistants into their daily development workflows, security skills with verification and structured output guarantees like this are likely to become standard components in the agent toolchain.

It's worth noting that the publicly available information currently comes primarily from the project description. Details about the specific implementation of the verification mechanism, the types of vulnerabilities supported, and real-world accuracy rates still await further documentation and practical validation.