Anthropic Research: LLMs Make Formal Software Verification at Scale Possible

Why Formal Verification Suddenly Matters

Software, after years of testing and repeated patching, still harbors critical vulnerabilities. The article notes that the Linux kernel has disclosed over 1,000 recent vulnerabilities, and more dangerously, frontier models are increasingly adept at chaining seemingly harmless small issues into exploitable attack chains. The ideal security process would be to find and fix every bug, then provide a mathematical proof that, under a given set of rules, no further provable vulnerabilities exist. This is precisely what formal verification does: build a mathematical model of the code and prove it always adheres to specified safety rules. Failure to prove often hints at a possible violating bug.

The cost has always been daunting. The article cites the figures: implementing GCEL4 takes roughly two person-years, while formal verification takes roughly 20 person-years. This cost has long confined formal verification to truly critical tasks like aviation. Anthropic's Frontier Red Team, in an official study published on October 22, 2026, proposes a turning point: large models can now take formal software verification from a niche expert task to producing proofs at scale at the RISC-V assembly level across hundreds of real open-source libraries, and in the process have uncovered bugs that went undetected for years.

From Testing to Proof: Solving the Specification Problem

Generating Specifications from Existing Test Suites

The hardest part of formal verification is writing specifications. Most programs lack a clear correctness definition like PNG or RFC. Anthropic's approach is to automatically convert existing compilable test suites into specifications—test suites that always pass, built on a general model of arbitrary RISC-V compiled binaries executing on Linux.

They start from Faust. Faust performs massive fuzzing with random compiled inputs against the same entry point to find crashes—essentially brute-force search over an input space that is nearly infinite, with no guarantee of covering all paths. They adapted Faust into a specification over compiled RISC-V instructions: any input smaller than 64MB bytes does not crash. If proven, then running Faust again on that entry point will no longer find crashes.

How the Proof Pipeline Runs

The full pipeline is: open-source library plus existing Faust tests, first compiled into a RISC-V executable, then layered with FreeRTOS's formal semantics for RISC-V and the OS plus the non-crash property, mechanically generating specifications, then Claude writes the proof, which is checked by the Rocq kernel. Green indicates it has been proven, assuming the trusted components are correct.

Faust (Fuzzing with Automated Unbounded Systematic Testing) is a class of automated testing frameworks based on fuzzing, whose core idea is to continuously inject random or semi-random inputs into program entry points and detect defects by monitoring signals such as crashes and assertion failures. Unlike formal verification, fuzzing can only cover paths actually executed and cannot exhaust the input space, so it can only prove "no crash was found," not "no crash exists." The cleverness of using the Faust test suite as a source of specifications in the article lies in: transforming the empirical goal "Faust will never find a crash" into the provable proposition "any input smaller than 64MB does not crash," thereby building a bridge between fuzzing and formal proof.

Rocq is the name adopted in 2025 after Coq proof assistant was renamed, an interactive theorem prover based on dependent type theory. It allows users to write mathematical definitions and theorems in a formal language and mechanically checks each step of the proof script's derivation via the kernel. Rocq's trusted base is small—as long as the kernel is correct, theorems checked by it hold. The proofs in the article are ultimately submitted to the Rocq kernel for checking, meaning that even if Claude generates an incorrect proof script, it will be rejected by the kernel.

FreeRTOS is a widely used open-source real-time operating system kernel; the article leverages the formal semantics of its RISC-V port to characterize OS-level behavior. Combining FreeRTOS's RISC-V formal semantics with the Linux process model serves to cover runtime assumptions such as system calls, memory layout, and thread scheduling in the proof, making the "non-crash" property meaningful in a real execution environment.

Scale Results and Bug Discovery

Officially, successful non-crash proofs cover 123 different OSS-Fuzz libraries across 239 entry points. These proofs total roughly 180 million lines of Rocq, with a typical proof around 600,000 lines and a median binary of about 18,000 RISC-V instructions. Runtime assumptions include single-threaded static binaries, fixed memory budget, RISC-V processor plus Linux; the trust boundary mainly consists of the RISC-V and Linux process formal models, the statement of the proven theorems, and the Rocq checker itself.

Bugs were also found and fixed during the proof process. Officially, the proof attempts uncovered bugs on the order of hundreds. To minimize interruptions, patches were allowed and specifications restated to continue proving. Examples include: an integer underflow in a decompressor causing out-of-bounds write, triggered only by a specific layout of about 20 megabytes; an out-of-bounds read in an image decoder, requiring about 64 megabytes of input and exceeding two levels of byte memory to trigger; an out-of-bounds write triggered only by a specific sequence of calls across two enormous specific orders; and the rediscovery of v202656132 on libxbat. There are also non-crash logic bugs, such as a feature limiting loadable files not always being enforced.

Trust Boundaries and Properties Beyond Non-Crash

A red line must be made clear here: formal verification does not equal no bugs. Formally verified does not equal bug-free. Remaining issues can only lie in trusted components or outside the scope of the proof. The current proofs assume single-threaded static RISC-V binaries, fixed memory budget, Linux; most people actually run on x86 or ARM plus dynamically linked glibc. The claimed scope is that on many OSS-Fuzz entry points, any input smaller than 64MB bytes does not trigger a crash, including ASAN, but ASAN does not detect all memory safety issues, nor does it cover all entry points of every library, dead code, or caller misuse—caller misuse requires the caller's own contracts.

Beyond non-crash, higher-level properties can also be proven. For example, round-trip encoding/decoding, algebraic laws, differential testing, etc. One C++ networking library in the article, to avoid recursive stack overflow when destructing standard library collections, needs to prove at the machine code level that the depth of libstdc++'s red-black tree is logarithmically bounded when balanced. The formal non-crash proof covers all paths reachable from the entry point.

ASAN (AddressSanitizer) is a compiler-instrumented memory error detection tool that can catch memory safety issues such as out-of-bounds access, use-after-free, and stack overflow at runtime. The proofs in the article include the condition "does not crash under ASAN," meaning the proof covers the error categories ASAN can detect, but ASAN itself is not complete—it cannot detect all undefined behavior, logic errors, or data races. Therefore, the conclusions of formal proofs need to be understood in conjunction with the boundaries of ASAN's detection capabilities: what is proven is "will not trigger the kind of crashes ASAN monitors," not "no security flaws exist whatsoever."

Costs Are Falling, Proofs Can Be Reused

A single proof can consume hundreds of millions of tokens, which must be reduced. Officially, on 36 small verification tasks, Opus 5.5 used about 40% fewer tokens than Opus when earning half credit. Proofs can also be reused: shared workspaces can copy each other's proofs, saving about 17% of tokens, and some tasks that previously failed individually were completed. In large-scale OSS-Fuzz, proofs heavily overlap on common parts such as the C runtime and ASAN initialization, with light modifications, heavy modifications, and new writes each accounting for a certain proportion. Analogy to MathLib: the formal mathematics library has accumulated nearly 300,000 theorems and over 700 contributors, and proofs can be reused.

The cost roadmap is general specifications plus binary-level verification, avoiding full-library rewrites, relying on maturity to first prove key shared libraries such as glibc, OpenSSL, and Curl, verifying once and reusing in many places.

MathLib is the formal mathematics library in the Lean theorem prover ecosystem, having accumulated nearly 300,000 theorems and over 700 contributors. Its significance lies in demonstrating the engineering feasibility of large-scale formal proofs: theorems can be referenced and reused by other proofs, and contributors can build on others' work without starting from scratch each time. The article uses MathLib as an analogy for the reuse prospects of formal software verification—when proofs of shared libraries (such as glibc, OpenSSL, Curl) are built once and referenced in many places, the verification cost of individual programs will be greatly amortized, consistent with the economic logic of theorem reuse in MathLib.

What This Means for Developers

The article's conclusion trajectory is: formal verification will move from hand-crafted expert work serving only the most critical tasks to being usable for almost all production software, like today's Fuzz. In an era where large models find holes at scale, processes that can guarantee safety properties are needed, and large-model scaled formal verification is one path toward broader guarantees. The article encourages more people to engage in formal verification with stronger models, especially developers who previously tried weaker models and found them inadequate.

A question worth pondering: if critical libraries could truly produce proofs at scale like Fuzz, what kind of dependency would you most want to add formal guarantees to first? Rust and Veil use Rust to write a Python bytecode interpreter and use Veil to prove it follows the CPython semantics described in an academic paper, empirically showing about 1.7 million random Python programs behave 99.95% consistently with CPython, with the remainder being known edge cases—such end-to-end correctness attempts may be exactly the starting point for the answer.

One-line Summary

Anthropic demonstrates that large models can generate formal non-crash proofs at the RISC-V assembly level for hundreds of open-source libraries and uncover latent vulnerabilities.

Paragraph Summary

The article introduces Anthropic Frontier Red Team's research: large models can now push formal verification from niche expert work toward scale. They automatically generate specifications from Faust test suites, establish a general model of arbitrary RISC-V binaries executing on Linux, have Claude write proofs checked by the Rocq kernel, successfully generating about 180 million lines of non-crash proofs for 123 OSS-Fuzz libraries and 239 entry points, discovering bugs on the order of hundreds in the process. The article also emphasizes that formal verification does not equal no bugs; the trust boundary centers on the RISC-V/Linux formal models, theorem statements, and the Rocq checker, with current assumptions of single-threaded, static binaries and fixed memory budget. Costs are falling, proofs can be reused, and the roadmap is general specifications plus binary-level verification, prioritizing key shared libraries like glibc, OpenSSL, and Curl.