Lean Verified Transformers
A research effort that applies the Lean theorem prover to formally verify the Transformer architecture. Starting from first principles, it proves key mathematical invariants of the Transformer and explores the application of formal verification methods within the field of deep learning.
Timeline (last 90 days)
The Lean Verified Transformers project uses the interactive theorem prover Lean to prove a series of Transformer invariants from scratch.
The author takes a speculative attitude in exploring the difficulty of extending the same rigorous proofs to the rest of the world's code, treating it as an exploratory thought experiment.
All Facts (2)
The Lean Verified Transformers project uses the interactive theorem prover Lean to prove a series of Transformer invariants from scratch.
50%UnverifiedThe author takes a speculative attitude in exploring the difficulty of extending the same rigorous proofs to the rest of the world's code, treating it as an exploratory thought experiment.
50%