[KongchangAI]
· 2 min read· 1,203 words

Turing Agent: A New Paradigm for Formal Verification of AI Mathematical Proofs

Turing Agent: A New Paradigm for Formal Verification of AI Mathematical Proofs

Turing Agent achieves end-to-end formal verification of research-level mathematics with a dual-scale agent architecture

The article introduces the technical report Turing Agent: Agentic Formal Verification for Mathematical Research, which proposes an end-to-end formal verification agent framework for research-level mathematics to address the trust crisis of the "era of proof abundance." Its core is a dual-scale architecture: a theorem formalization agent precisely translates natural language propositions into formal statements, while a proof formalization agent generates and completes proofs, supported by structured decomposition, parallel verification, accumulated experience, and selective semantic review. Using theoretical computer science as the testbed, Turing Agent achieved roughly 57-62% faithful formalization success on Formal TCS Bench and 83% pass@1 on the RLMEval TCS subset, outperforming general coding agents and verifying three previously unsolved TCS tasks from OpenAI's collection.

Large language models are rapidly advancing in the field of mathematical discovery, moving from benchmark problems toward frontier research questions. This brings a new challenge known as the "era of proof abundance": how do we ensure that the massive number of AI-generated proofs are correct and transform them into trustworthy, machine-verifiable mathematical knowledge?

A recently released technical report, Turing Agent: Agentic Formal Verification for Mathematical Research, presents a new solution. Turing Agent is an agent framework designed to orchestrate end-to-end formal verification, specifically targeting research-level mathematics. Rather than having a model directly generate proofs, it uses a dual-scale architecture that integrates theorem formalization and proof formalization agents, combined with structured proof decomposition, parallel verification, accumulated proof experience, and selective semantic review to systematically improve verification success rates.

Turing Agent Project Overview

Core Method: Dual-Scale Agent Architecture

The design philosophy of Turing Agent returns to the essence of verification—gradually translating mathematically stated theorems and proofs in natural language into computer-checkable formal code. Its architecture comprises two key levels:

  • Theorem Formalization Agent: Responsible for transforming mathematical propositions from research papers into precise formal statements, avoiding ambiguity.
  • Proof Formalization Agent: Building on theorem formalization, it generates or completes formal proofs, and uses structured decomposition to break long proofs into manageable subgoals.

The two work in concert, together with parallel verification and an accumulated proof experience library, enabling the system to handle mathematical content with complex dependencies and deeply nested quantifiers. Selective semantic review is used to check for potential semantic drift during formalization, ensuring that the transformed propositions match the original intent.

Formal verification refers to rewriting mathematical propositions and proofs in machine-checkable formal languages (such as Lean, Coq, and Isabelle), then having a proof assistant verify each step of reasoning, thereby eliminating errors at the logical level. Unlike natural language proofs, every step of a formal proof must be accepted by the system, so the results are highly trustworthy—but the cost is extremely high authoring effort. Theorem formalization focuses on accurately translating natural language propositions into formal statements, with the difficulty lying in eliminating ambiguity and filling in implicit assumptions; proof formalization involves constructing or completing a formal proof given a statement, typically requiring decomposition into numerous auxiliary lemmas. Together they constitute the end-to-end verification pipeline and form the basis for the dual-scale architecture division in this work.

Agent here refers to a large language model system capable of autonomous planning, tool invocation, and iterative correction based on feedback. Unlike one-shot generation, an agent reads compiler error messages when verification fails, adjusts its proof strategy, and retries—transforming formal verification from single-round generation into a multi-round search and repair process. Structured proof decomposition draws on the "divide and conquer" approach in mathematics: breaking a long proof into several independently verifiable subgoals, formalizing and verifying them in parallel, and finally combining them. This decomposition not only reduces the difficulty of a single generation but also allows the number of auxiliary declarations and their dependencies to be explicitly managed by the system—a key engineering technique for handling research-level long proofs.

Why Theoretical Computer Science as a Testbed

The research team collaborated with researchers in theoretical computer science (TCS) and focused evaluation on TCS. This is an extremely demanding field for formal verification: problems typically involve intricate quantifier structures, complex dependencies, and strict semantic requirements. Using TCS as a touchstone tests the robustness of Turing Agent in real research-level mathematics.

In this setting, the system faces open problems from frontier papers rather than textbook exercises. This requires the agent to understand not only mathematical notation but also the macro structure and micro details of proofs.

Theoretical computer science (TCS) studies the theoretical foundations of computation, with typical topics including complexity classes, reductions, lower bounds, and algorithm correctness. Its propositions often take the form of multi-level quantifier nesting such as "for all graphs G, there exists... such that...". Such structures are extremely unfriendly to formalization: a slight deviation in quantifier order changes the meaning of the proposition; proofs often rely on a long chain of lemmas, and many steps are implicitly omitted in papers. Therefore, TCS problems effectively test a formalization system's ability in semantic fidelity and dependency management. pass@1 is an evaluation metric referring to the proportion of correct answers a model obtains when generating only once, reflecting single-shot reliability more strictly than best-of-many sampling; faithful formalization success rate emphasizes that the formal statement is semantically consistent with the original proposition, not merely that it compiles.

Performance: Significantly Outperforming General Coding Agents

According to the report, Turing Agent achieved approximately 57-62% faithful formalization success rate on Formal TCS Bench and an overall pass@1 of 83% on the TCS-focused subset of RLMEval. The tests used models including GLM-5.3 and DeepSeekAd-V4-Pro.

By comparison, direct generation and general coding agents (such as Codex CLI and Claude Code) performed markedly worse on the same tasks. This indicates that a specialized agent architecture for mathematical formal verification can deliver qualitative improvements.

More strikingly, a Turing Agent built on DeepSeek successfully completed formal verification of three previously unsolved TCS tasks from OpenAI's October 2026 collection. One particularly difficult TCS problem required approximately one hundred formally verified auxiliary declarations to complete. This demonstrates the system's ability to handle long-chain, multi-dependency proofs.

Byproducts: Simpler Proofs and Reusable Knowledge

The value of Turing Agent goes beyond verifying correctness. By actively exploring and refining the formalization process, it can discover simpler alternative proofs, reveal unnecessary dependencies in the original proof, and generate reusable mathematical knowledge during verification. These byproducts have potential value for mathematical research itself—AI can not only verify what is known but may also help mathematicians optimize proofs and clarify logical structures.

This capability transforms formal verification from "post-hoc checking" into a "research assistant". As the number of AI-generated proofs grows, this automated, trustworthy verification layer will become indispensable. Turing Agent demonstrates a viable path: using agents to orchestrate formal verification, turning unreliable natural language proofs into machine-verifiable knowledge.

For readers interested in AI for Math, the significance of this work lies in directly confronting the trust crisis brought by "proof abundance" and providing an engineering-scalable, measurably effective answer.

One-line Summary

Turing Agent achieves end-to-end formal verification of research-level mathematics with a dual-scale agent architecture

Paragraph Summary

The article introduces the technical report Turing Agent: Agentic Formal Verification for Mathematical Research, which addresses the trust crisis of the "era of proof abundance" brought by the surge in AI-generated proofs by proposing an end-to-end formal verification agent framework for research-level mathematics. Its core is a dual-scale architecture: a theorem formalization agent precisely translates natural language propositions into formal statements, while a proof formalization agent generates and completes proofs, supported by structured proof decomposition, parallel verification, accumulated proof experience, and selective semantic review. The team chose theoretical computer science as the testbed because of its deep quantifier nesting, complex dependencies, and strict semantic requirements. Results show that Turing Agent achieved approximately 57-62% faithful formalization success rate on Formal TCS Bench and 83% pass@1 on the TCS subset of RLMEval, clearly outperforming general coding agents, and successfully verified three previously unsolved TCS tasks from OpenAI's collection. The verification process can also discover simpler alternative proofs, expose unnecessary dependencies, and generate reusable knowledge.

#Turing Agent#formal verification#mathematical research#dual-scale architecture#theoretical computer science#AI for Math
Share:

Related articles