LFortran + Enzyme: A New Path to Automatic Differentiation for Fortran Code

A Technical Collision Spanning Half a Century

Fortran (Formula Translation) was born in 1957, released by a team led by IBM engineer John Backus. It was the world's first widely adopted high-level programming language — one that allowed programmers to write code resembling mathematical formulas, without directly manipulating machine instructions. Through successive standard versions including FORTRAN 77, Fortran 90/95, Fortran 2003, and Fortran 2018, modern Fortran now supports object-oriented programming, parallel computing (Coarray), and more.

To this day, it remains irreplaceable in scientific computing, climate simulation, fluid dynamics, and high-performance numerical computation. The world's top weather forecasting systems (such as ECMWF's IFS model), nuclear weapons simulation codes (such as legacy codebases at U.S. national laboratories), and numerous CFD (Computational Fluid Dynamics) solvers still use Fortran as their core — code that has often been tuned and validated over decades, with numerical accuracy and performance that cannot easily be replaced.

Yet as modern AI and optimization algorithms increasingly depend on Automatic Differentiation (AD) as a core technology, the aging Fortran ecosystem seems somewhat out of place. Recently, work combining the LFortran compiler with the Enzyme automatic differentiation framework has attracted attention in the technical community, aiming to give Fortran the wings of differentiable computing.

What Is Automatic Differentiation, and Why Does It Matter

Automatic differentiation is the technical cornerstone of modern machine learning frameworks like PyTorch, TensorFlow, and JAX. It computes derivatives (gradients) of arbitrary programs precisely and efficiently — without manually deriving mathematical formulas, and unlike finite differences, without accumulating numerical errors.

The essence of AD is decomposing a program into a sequence of elementary operations (addition, subtraction, multiplication, division, trigonometric functions, etc.), then systematically combining the derivatives of each step via the Chain Rule. AD comes in two core modes: Forward Mode propagates derivatives from inputs to outputs (Jacobian-vector product), suited for cases where input dimensionality is far smaller than output dimensionality; Reverse Mode (Backpropagation) propagates gradients from outputs back to inputs (Vector-Jacobian product), suited for cases where output dimensionality is far smaller than input dimensionality — this is precisely the theoretical foundation of the backpropagation algorithm in deep learning. In scientific computing, when model parameter counts reach into the millions, the computational cost of reverse mode is comparable to a single forward pass, making large-scale parameter optimization computationally feasible.

Comparing Three Differentiation Approaches

Manual differentiation: Deriving derivative formulas by hand — precise but highly error-prone and difficult to maintain for complex code.
Numerical differentiation: Approximating via finite differences — simple to implement but low accuracy and high computational overhead.
Automatic differentiation: Decomposing program operations via chain rule — combining precision with efficiency, the best of the three approaches.

For scientific computing, gradient information is critical in parameter optimization, sensitivity analysis, inverse problem solving, and data assimilation. In climate modeling, for instance, researchers need to understand how output results respond to thousands of input parameters — this is precisely where automatic differentiation delivers its core value.

LFortran: A Modern Open-Source Fortran Compiler

LFortran is an ambitious open-source project aimed at building a modular, modern Fortran compiler. Compared to traditional compilers like GFortran or Intel's compiler, it has several distinctive characteristics:

Interactive execution: Supports a Jupyter Notebook-style interactive mode similar to Python, enabling "write-and-run" Fortran.
LLVM-based: Built on the LLVM compiler infrastructure at its core, capable of generating highly efficient machine code.
Intermediate Representation (ASR): Features a semantically rich Abstract Semantic Representation, facilitating code analysis and transformation.

It is precisely LFortran's LLVM foundation that provides the basis for its integration with Enzyme.

Enzyme: Automatic Differentiation at the LLVM Level

LLVM (Low Level Virtual Machine) was originally initiated by Chris Lattner at the University of Illinois in 2003 and has since become one of the most important open-source compiler infrastructures in the industry. Its core design philosophy decouples the compilation process into three independent stages: Frontend → Intermediate Representation (IR) → Backend. LLVM IR is a typed, SSA (Static Single Assignment) form low-level language that is both readable and analyzable. Thanks to this decoupled architecture, many languages — including Clang (C/C++), Rustc (Rust), Swift, and Julia — all use LLVM as their backend. LLVM's Pass mechanism allows developers to insert custom transformations at any stage of the compilation pipeline, and this is precisely how Enzyme operates.

Enzyme was developed by William Moses and colleagues at MIT (Massachusetts Institute of Technology), attracting widespread attention around 2020. Its distinctive feature is that it differentiates directly at the LLVM Intermediate Representation (IR) level, rather than operating at the source code level. Its core innovation is treating automatic differentiation as a compiler transformation: given an LLVM IR function, Enzyme analyzes its data flow and control flow to automatically generate a corresponding adjoint function, implementing reverse-mode AD.

This means Enzyme can perform automatic differentiation on any language that compiles to LLVM IR — whether C, C++, Rust, or Fortran code compiled through LFortran. Because it executes differentiation after compiler optimization, the generated derivative code often achieves superior performance. Benchmarks show that gradient code generated by Enzyme is competitive with hand-written derivatives and in some cases even surpasses expert manual implementations. Enzyme currently supports multiple language ecosystems including C, C++, Fortran (via LFortran/Flang), Rust, Julia, and MLIR.

Three Key Advantages of IR-Level Differentiation

Language agnosticism: Anything that can generate LLVM IR can be differentiated — no need to rewrite the AD engine separately for each language.
High performance: Executing differentiation on already-optimized IR effectively avoids redundant computation common in source-level AD.
Low invasiveness: Existing Fortran code requires virtually no rewriting to gain full gradient computation capability.

LFortran + Enzyme: Making Fortran Gradient Computation a Reality

The core idea behind this work is elegantly simple: LFortran compiles Fortran source code into LLVM IR, which is then handed off to Enzyme to automatically generate corresponding derivative code at the IR level. In this way, the vast body of legacy Fortran scientific computing code can theoretically gain the gradient computation capabilities required by modern AI workflows — without large-scale rewrites.

This path carries profound significance for the scientific computing community. Previously, integrating a physics simulation model written in Fortran into a gradient-based optimization or machine learning pipeline often required enormous migration costs — either rewriting in Python or manually deriving derivatives. Through the combination of LFortran and Enzyme, the gap between traditional scientific software and modern AI can potentially be genuinely bridged.

Potential Significance and Practical Challenges

Technical Value: A Bridge to Scientific Machine Learning

Scientific Machine Learning (SciML) is a rapidly emerging interdisciplinary research direction dedicated to deeply integrating physical laws, numerical methods, and machine learning. Physics-Informed Neural Networks (PINNs), systematically proposed by George Karniadakis' team at Carnegie Mellon University in 2019, embed partial differential equation (PDE) constraints into neural network loss functions, enabling networks to satisfy physical conservation laws while fitting data. Another important paradigm is Differentiable Simulation — directly differentiating through numerical simulators and using gradient descent to optimize simulation parameters. Both paradigms fundamentally require "differentiability" — and this is precisely the greatest shortcoming of existing Fortran codebases.

Activating legacy code assets: Decades of accumulated Fortran scientific code can directly participate in differentiable computing without starting from scratch.
Critical bridge to Scientific Machine Learning (SciML): Provides a practical toolchain for "physics-informed machine learning" that fuses physical models with neural networks.
Advancing the open-source compiler ecosystem: Powerfully demonstrates the unique value of LLVM as a universal backend for cross-language automatic differentiation scenarios.

Engineering Challenges

This project is still in its early exploratory stages, and several deep technical challenges remain for real-world engineering deployment:

Multidimensional arrays are Fortran's most fundamental data structure. Their column-major storage order differs from C's row-major order, requiring correct handling of stride information when generating IR — otherwise differentiation results will produce array access errors. Pointer aliasing is particularly complex in Fortran: the Fortran standard assumes by default that array arguments do not alias (which is historically one reason Fortran numerical performance surpasses C), but Enzyme's Activity Analysis needs precise knowledge of memory access patterns to safely generate adjoint code. Furthermore, Fortran's parallel extensions (OpenMP, Coarray Fortran) introduce shared-memory and message-passing semantics, placing additional demands on the AD framework's concurrency safety. These are all issues the LFortran+Enzyme combination must systematically resolve before reaching production-grade applicability.

Conclusion

Combining Fortran — born in 1957 — with cutting-edge automatic differentiation technology is itself an act of remarkable engineering imagination. It embodies a pragmatic technical philosophy: rather than discarding scientific code validated over decades, use modern tools to breathe new life into it.

The fusion of LFortran and Enzyme may well be a key that unlocks the connection between the worlds of "traditional high-performance computing" and "modern AI optimization." Despite the challenges ahead, this direction is undoubtedly worth sustained attention from practitioners at the intersection of scientific computing and AI.