Why Did the IBM Spyre Accelerator Abandon Its In-House Compiler Stack for PyTorch Inductor?

IBM's Spyre AI accelerator recently made a key decision: switching the torch.compile backend from its in-house sendnn compiler stack to the open-source community-maintained Inductor. This isn't just about swapping one tool for another—it touches on a classic dilemma in compiler engineering: build it yourself or reuse what exists.

From sendnn to Inductor: A "Tear Down and Replace" of the Compiler Stack

According to the official PyTorch blog, the software stack for IBM's Spyre accelerator previously relied on sendnn—a fully independent compilation stack built from scratch. This approach could quickly get the pipeline running in the early days, but as model complexity grew and hardware iterations accelerated, maintenance costs began spiraling out of control.

After switching to Inductor, Spyre plugs directly into the PyTorch mainline ecosystem. Inductor is the default backend of torch.compile, capable of lowering PyTorch computation graphs into Triton kernels or C++/OpenMP code, then performing code generation for the target hardware. This means IBM no longer needs to reinvent the wheel, and can instead focus its efforts on the Spyre-specific hardware adaptation layer.

This path aligns with the strategies of vendors like Intel and AMD: integrate with Inductor's extension mechanisms, perform hardware-specific optimizations through custom backend interfaces, rather than maintaining a parallel compilation pipeline.

The Hidden Costs of a Self-Built Compiler Stack

The core complexity of a compiler isn't about "getting code to run"—it's about "getting code to run well on every generation of hardware." A self-built stack means implementing and debugging everything yourself: operator fusion strategies, memory planning, loop transformations, backend code generation. When the PyTorch upstream operator set expands every quarter, the cost of keeping up rises exponentially.

Inductor provides a set of lowering and optimization passes validated across a large number of models, and hardware vendors only need to implement a small number of interfaces to gain community-level optimization capabilities. For IBM, the math was clear.

torch.compile is the just-in-time compilation (JIT) mechanism introduced in PyTorch 2.0. Developers simply add a single @torch.compile decorator to the outer layer of a model, and at runtime the computation graph executed by Python is captured, passed through multiple layers of graph optimization, and handed to a backend to generate efficient kernels. Its workflow has three steps: first, TorchDynamo captures the computation graph at the bytecode level; then AOTAutograd handles the graph structure for both forward and backward passes; finally, a backend compiler lowers it into executable code. torch.compile itself is just a "dispatch layer"—the actual code generation capability depends on the backend it's attached to. The default backend is Inductor, so "switching backends" is essentially replacing torch.compile's final output stage without changing the frontend capture logic. Understanding this layering is the premise for understanding this article: what IBM switched is the output, not the entire compilation framework.

Inductor's Extension Points: How Hardware Vendors Plug In

Inductor's design allows third-party hardware to intervene in the compilation process through multiple mechanisms.

Custom Backends and Triton Code Generation

The most direct approach is to implement Inductor's backend interface. When generating kernels, the compiler can choose either the Triton path or the C++ path. For accelerators with custom programming models, vendors can take over code generation for specific operators, mapping Inductor's intermediate representation onto their own instruction set.

This is the route Spyre took. IBM built Spyre-specific scheduling and memory hierarchy adaptation on top of Inductor, rather than starting from scratch.

Triton is a programming language and compiler for GPU kernels open-sourced by OpenAI. It lets developers write parallel kernels using near-Python syntax, while the Triton compiler automatically handles low-level optimizations such as thread partitioning, shared memory allocation, and vectorization. When generating kernels, Inductor by default translates the optimized computation graph into Triton code, then hands it to the Triton compiler to produce machine code; when the target hardware doesn't support Triton or isn't suited to it, it falls back to the C++/OpenMP path. For hardware vendors, this means that as long as they implement the lowering rules from graph to Triton (or C++), they can borrow the large collection of kernel templates accumulated by the community, without rewriting from PTX or assembly. Spyre's adaptation was built precisely on this layer of abstraction: IBM only needs to map Inductor's intermediate representation onto the Spyre instruction set, while the heavy lifting of frontend capture and graph optimization is entirely handled by the PyTorch mainline.

The Feedback Loop of Upstream Contributions

Plugging into the open-source stack has another often-underestimated benefit: changes can flow back upstream. When IBM discovers during adaptation that a particular Inductor pass doesn't support sparse computation well enough, it can submit a PR to improve the general logic, rather than patching a private fork. This kind of feedback makes subsequent adaptation increasingly lightweight.

"Upstream" in open-source collaboration refers to the project's mainline codebase, as opposed to a fork maintained by a vendor. Submitting a PR to the mainline and having it merged means your change becomes a general capability shared by all users. The problem with a private fork is that every time the mainline updates, you have to merge upstream changes back in; conflicts accumulate, and eventually you may be unable to keep up at all. The benefit of the feedback mechanism is bidirectional—once a general improvement contributed by a vendor (for example, support for sparse operators, specific data types, or new hardware features) enters the mainline, all subsequent adaptation work stands on a higher baseline. This explains why vendors like Intel, AMD, and IBM are willing to dedicate engineers full-time to upstream contributions: the short-term cost of PRs buys a systematic reduction in long-term maintenance burden, and also prevents each vendor's private patches from fragmenting the common ecosystem into mutually incompatible islands.

What This Means for the AI Chip Ecosystem

The signal sent by this switch is more valuable than the technical details.

Fragmented compiler stacks are converging. Over the past few years, nearly every AI chip company has built its own compiler, resulting in extremely high model porting costs and forcing users to tune separately for each chip. Inductor is becoming the de facto intermediate layer, reducing "chip-framework" adaptation from O(M×N) to nearly O(M+N).

The performance ceiling of open-source backends is rising. In the early days, Inductor's performance on non-NVIDIA hardware was often criticized, but with sustained investment from Microsoft, Intel, AMD, and now IBM, the quality of general code generation can already cover a substantial portion of training and inference workloads. The performance advantage of a self-built stack in specific scenarios may not offset its maintenance disadvantage.

The competitive focus of hardware vendors is shifting. As the compilation layer becomes unified, differentiation falls to the chip architecture itself, interconnect bandwidth, memory bandwidth, and the genuinely unique parts of the software stack—such as Spyre's specific dataflow scheduling. This is good news for users: switching chips no longer means rewriting the entire toolchain.

O(M×N) to O(M+N) is a classic framework in software engineering for describing the "adaptation combinatorics explosion." Suppose there are M AI chips and N mainstream frameworks. If every "chip-framework" pair requires separately developing and maintaining a set of compilation adaptations, the total workload is M×N, expanding multiplicatively as both sides grow. Once a widely accepted intermediate layer (such as Inductor) emerges, chip vendors only need to adapt to that layer, and framework developers only need to integrate with that layer, reducing the workload to the additive relationship M+N. This is precisely the pattern repeatedly validated in domains like operating systems, browsers, and database drivers: an intermediate abstraction layer is the key means of controlling ecosystem complexity. For AI chips, whoever gets incorporated into the mainstream intermediate layer is more likely to be supported by developers by default; and the intermediate layer itself has its evolution direction led by the upstream community—which is the deeper motivation for vendors racing to invest upstream.

Risks Worth Watching

As a general-purpose backend, Inductor still has limitations in extreme optimization scenarios. For workloads that are extremely latency-sensitive or highly customized at the operator level, general passes may not match a specialized compiler. IBM needs to retain handwritten optimization paths for Spyre's key operators while integrating with Inductor; otherwise, it may lose competitiveness on certain benchmarks.

Another issue is upstream compatibility. Inductor iterates quickly, with interfaces and internal IR changing from time to time. A vendor's adaptation layer needs to keep up continuously—which is itself a maintenance cost, just shifted from "maintaining the entire compiler" to "maintaining the adaptation layer plus tracking upstream."

Overall, IBM's move is a pragmatic engineering trade-off. At a stage where AI chip compilers are far from standardized, choosing to stand on the side of the PyTorch ecosystem at least ensures that developers don't have to learn a new toolset for Spyre.