Google TPU: Five Generations Decoded — From v2 to Ironwood, a 30x Leap in Energy Efficiency

A Single Paper Unveils Google's Five-Generation Supercomputer Evolution

Google's Jeff Dean recently shared a landmark paper on social media — co-authored by senior Google researchers Norm Jouppi, Sridhar Lakshmanamurthy, Cliff Young, and David Patterson, set to be published in the July/August 2026 issue of IEEE Micro. Titled "Google's Training Supercomputers from TPU v2 to Ironwood: Architectural Stability, Scale, Resilience, Power Efficiency, and Sustainability Across Five Generations," the paper systematically traces the architectural evolution of Google's training supercomputers across five generations, from TPU v2 to the latest Ironwood.

This paper is far more than a list of hardware specs. It offers a deep look into the consistency of Google's AI chip design philosophy and the engineering strategies deployed in response to the dramatic shifts in AI workloads.

Key Evolutionary Threads Across Five TPU Generations

From 256 to 9,216 Chips: Exponential Scale Expansion

One of the most striking data points in the paper is the change in chip count per Pod (a compute cluster unit). In the TPU v2 era, a single Pod contained just 256 chips; by the time of the latest Ironwood, that number has surged to 9,216. This means that in less than a decade, Google has increased the chip density of a single compute cluster by roughly 36x.

This scaling is far from simple stacking. As chip counts grow, challenges in interconnect topology, fault tolerance, and power management increase nonlinearly — and this is precisely one of the core topics the paper addresses.

A Fundamental Shift in Cooling: From Air to Liquid

TPU v2 relied on traditional air cooling, but starting with TPU v3, Google made a full transition to liquid cooling (water cooling). In hindsight, this shift seems almost inevitable — as chip power consumption and cluster density continued to climb, air cooling could no longer meet the thermal demands of high-density deployments.

Liquid cooling not only dramatically improved thermal efficiency but also enabled more compact physical layouts, directly paving the way for subsequent TPU generations to pack more compute power into less space.

Interconnect Architecture: From 2D Torus to 3D Torus

Chip-to-chip interconnect topology is the lifeblood of large-scale AI training. The paper notes that Google's TPU interconnect evolved from an early 2D Torus to a 3D Torus architecture. The 3D Torus topology offers significant advantages in large-scale parallel computing:

• Shorter average communication paths
• Higher bisection bandwidth
• Stronger fault tolerance characteristics

This design choice reflects Google's long-term commitment to custom interconnects — unlike competitors who rely on general-purpose solutions such as InfiniBand, Google opted for a custom network topology deeply co-designed with the TPU.

The Energy Efficiency Revolution: 30x Reduction in Power per FLOP

Perhaps the most eye-catching metric in the paper is the 30x improvement in TFLOPS/Watt (compute per watt). Spanning five TPU generations, this figure represents an extraordinarily significant advance in computational energy efficiency.

This 30x efficiency gain stems from coordinated optimization across multiple layers:

• Process technology advances: Transistor-level efficiency improvements from more advanced semiconductor nodes
• Architectural optimization: Specialized compute units designed for dominant models like Transformers
• System-level engineering: Holistic improvements encompassing liquid cooling, power management, and interconnect efficiency
• Hardware-software co-design: Continuous improvements in compilers and scheduling systems

At a time when AI training's energy consumption is drawing increasing scrutiny, this data carries significant industry implications. The paper's title deliberately includes "Sustainability," signaling that Google has elevated energy efficiency to a core chip design objective.

Workload Evolution: The Transformer Takes Over

Another important dimension of the paper is its description of how AI workloads within Google have shifted over time. As Jeff Dean noted — "lots more transformer-based models" — Transformer-based models now hold absolute dominance across Google's training tasks.

This workload transformation has, in turn, profoundly influenced the direction of TPU architecture design. From the TPU v2 era, when hardware needed to accommodate diverse model architectures like CNNs and RNNs, to today's heavy optimization for Transformer matrix multiplications and attention mechanism computations, the center of gravity in hardware design has fundamentally shifted.

Architectural Stability: An Underappreciated Design Philosophy

"Architectural Stability" in the paper's title is an easily overlooked yet critically important keyword. Maintaining architectural coherence across five generations of rapid iteration means:

• Software ecosystem continuity: Developers don't need to rewrite training frameworks for each new hardware generation
• Accumulated operational expertise: Large-scale cluster management knowledge can be reused across generations
• Supply chain predictability: A stable architectural direction facilitates long-term planning

This strategy of "incremental innovation" stands in sharp contrast to approaches that pursue radical architectural overhauls with every generation. Google's practice demonstrates that at hyperscale deployment, architectural stability is itself a competitive advantage.

Resilience by Design: The Lifeline of Large-Scale Systems

When a training cluster contains nearly ten thousand chips, hardware failure is no longer a question of "if" but of "when" and "how often." The paper devotes specific attention to the Resilience dimension, which is essential for understanding the engineering realities of large-scale AI training.

In systems composed of thousands of chips, chip failures, memory errors, and interconnect disruptions are practically routine. How these failures are handled without interrupting training tasks directly determines the system's actual utilization and training efficiency.

Summary and Outlook

This paper provides the industry with a rare window into nearly a decade of Google's technical accumulation and design thinking in the domain of AI-specific chips. The five-generation evolution from TPU v2 to Ironwood represents not just simple growth in compute power and scale, but a systematic engineering methodology:

1. Achieving continuous innovation on a foundation of architectural stability
2. Elevating energy efficiency and sustainability to core design objectives
3. Adapting hardware direction based on actual workload changes
4. Full-stack systematic optimization from chip to cluster

Against the backdrop of NVIDIA GPUs dominating the AI training market, Google's TPU evolution offers the entire industry an important alternative reference point. As the paper is officially published in IEEE Micro, it is sure to spark broader technical discussion.