Fighting AI Code Slop: Formal Verification Is Becoming the New Weapon

As AI code slop spreads, Lean and AI are turning formal verification from an academic weapon into an engineering defense line.
The article discusses the hidden risks of "code slop" from AI-generated code: syntactically correct code that runs but hides boundary errors and unhandled branches, which traditional testing can't cover. Formal verification uses mathematical proof instead of sampling tests to guarantee properties under all legal inputs, but was long confined to safety-critical fields due to high specification and proof costs. Lean lets specs and implementations share one framework, AI lowers the proof barrier, and layered verification fits engineering reality. As code producers shift from humans to models, verification must shift from human inspection to machine proof—though specification accuracy, proof debugging, CI integration, and talent scarcity remain challenges.
The flood of AI-generated code is eroding software engineering with low-quality "slop." A weapon once confined to mathematicians and academia—formal verification—is being brought into mainstream development workflows by AI and the Lean language.
The AI Code Slop Dilemma
Large models write code faster and faster, but problems follow: the code looks syntactically correct and the logic runs, yet it hides subtle boundary errors, unhandled branches, and undiscovered assumptions. This kind of "code slop" isn't an explicit crash—it's a long-term hidden risk. Traditional testing can only cover the scenarios you think of, and is powerless against "situations you didn't think of."
According to the original article, the core value of formal verification lies in replacing sampling-based testing with mathematical proof—not running a few test cases to see if anything errors out, but proving that under all legal inputs, the program satisfies the specified properties.
Why Formal Verification Was Once Niche
Proving software correctness mathematically has traditionally required expert-level investment: writing formal specifications, constructing proofs, and maintaining proof scripts—high barriers and long cycles. The original article notes that this work has always been the domain of specialized researchers, and ordinary engineering teams struggle to take it on.
The difficulty isn't just in writing proofs, but in the maintenance cost of proofs—change the code and the proof may become invalid, requiring re-derivation. This has kept formal verification confined for a long time to a few fields like safety-critical systems and aerospace.
Formal Specification is the starting point of formal verification, referring to the precise description of what a program "should do" rather than "how to do it" in mathematical language. Common forms include preconditions, postconditions, loop invariants, and data type invariants. The difficulty of specification lies in this: it must capture the intent of requirements unambiguously, while natural-language requirements themselves often carry implicit assumptions. Get the specification wrong and even the most beautiful proof is meaningless—this is the so-called "garbage in, garbage out." Unlike testing, once a specification is written, it can be checked by machine across all legal inputs, so specification quality directly determines verification value. In engineering practice, specifications often require collaboration between domain experts and formal methods experts—one reason the barrier is so high.
How AI and Lean Are Changing the Game
The turning point comes from the combination of two forces. The first is interactive theorem provers like Lean, which are simultaneously a programming language and a proof assistant, allowing specifications and implementations to be expressed in the same framework. The second is that AI models are beginning to participate in proof construction—models can propose lemmas, search proof paths, and complete tedious proof steps, automating what once required experts working by hand.
The original article's description: AI and Lean are helping automate this process. This means the entry barrier for formal verification is being significantly lowered, moving from "expert-exclusive" toward "team-usable."
A key shift in thinking: verification doesn't have to happen all at once. You can start by formalizing core invariants, key data structures, and high-risk paths, while the rest continues to be backstopped by testing. This layered strategy brings formal verification closer to engineering reality.
Interactive Theorem Prover is a class of human-machine collaborative proof tools, with Lean, Coq, and Isabelle as representatives. Users write proof steps, and the tool checks each one for logical rigor, accepting no "obviously true" leaps. Lean's special feature is that it is also an executable programming language: the same code can both compile and run, and be treated as a mathematical object to prove properties about, enabling specifications and implementations to be expressed in the same framework. Unlike SMT tools that rely on automated solvers, interactive proofs can handle more complex inductive structures and mathematical abstractions, but require more human guidance. AI's intervention targets precisely this human-guidance step—models can generate proof drafts, search lemma libraries, and fill in mechanical steps, letting people focus their energy on key insights.
Why This Matters Especially for AI-Generated Code
AI writes code far faster than humans can review it. Code review can catch obvious style issues and logic holes, but cannot prove correctness. Formal verification provides a machine-checkable correctness guarantee—if the proof passes, it passes, independent of the reviewer's experience and attention.
In other words, when the producer of code shifts from human to model, the verification method must also upgrade from "human inspection" to "machine proof." This constitutes a structural defense line against code slop.
Code Review is currently the main human defense line for software quality, relying on the reviewer's experience, attention, and contextual understanding. Its fundamental limitation: reviewers can only check problems they think of, cannot exhaustively enumerate all input paths, and cannot prove the "absence" of a certain class of errors. For code generated by large models, reviewers also face the misleading "looks reasonable" effect—models excel at producing syntactically correct, stylistically coherent code, making defects more hidden. Formal verification and code review are not substitutes: review excels at catching deviations from design intent, maintainability issues, and team convention problems, while formal proof excels at guaranteeing that specific properties hold under all inputs. Only by combining the two can you cover the different blind spots of "what humans didn't think of" and "what machines didn't think of."
Challenges Still Facing Adoption
The progress in automated proving is noteworthy, but the original article does not claim the problem is solved. Real-world adoption still faces several hurdles: how to write accurate formal specifications, how to determine when a proof fails whether the code is wrong or the specification is wrong, and how to integrate proof engineering with existing CI pipelines. None of these can be sidestepped by models alone.
Another practical issue is talent—engineers who understand both formal methods and the business domain remain scarce. AI lowers the cost of proof construction, but does not eliminate the need for the judgment involved in "translating requirements into specifications."
Signals Worth Watching
Formal verification's move from academic tool to engineering weapon is a clear signal of software quality governance in the AI era. Its significance isn't in replacing testing, but in filling the gap testing cannot cover—the guarantee for "all possible cases." For teams relying on AI-generated code, the earlier they understand this methodology, the better they can hold the quality bottom line as code volume swells.
One-line Summary
As AI code slop proliferates, Lean and AI are turning formal verification from an academic weapon into an engineering defense line.
Paragraph Summary
The article discusses the hidden risks of "code slop" brought by AI-generated code: code that is syntactically correct and whose logic runs, yet hides boundary errors and unhandled branches, while traditional testing can only cover scenarios you think of. Formal verification replaces sampling-based testing with mathematical proof, guaranteeing that under all legal inputs the program satisfies specified properties—but historically it was confined to safety-critical fields due to the high cost of writing specifications, constructing proofs, and maintaining them. The Lean theorem prover lets specifications and implementations be expressed in the same framework, AI models participate in proof construction to lower the barrier, and layered verification strategies are closer to engineering reality. The article emphasizes that when the producer of code shifts from human to model, the verification method must also upgrade from "human inspection" to "machine proof," though specification accuracy, proof debugging, CI integration, and talent scarcity remain adoption challenges.
Related articles

Inkling 2 Mini Released: A New Contender in Small Models?
Thinking Machines releases the small model Inkling 2 Mini, sparking heated discussion in the Hacker News community. This article analyzes the release, community reaction, and the small model trend.

Will GTA VI Be the Last "Purely Human" AAA Blockbuster?
What does calling GTA 6 "the last game of its kind" really mean? This article interprets the video's view: it refers to a historic work built by thousands of people without AI, with the full force of an entire company—not the end of big-budget games—using the Ocarina of Time remake as a counterexample of low cost and high quality.

Inkling 2 Mini Lands on Ollama: How a 5.4B Model Achieves Deep Reasoning
ThinkyMachines' Inkling 2 Mini is now on Ollama's pre-release. With only 5.4B activated parameters, it achieves deep reasoning by cycling through middle layers three times per token, runs locally on a 32GB Mac, and targets coding, agents, and vision tasks.