Google Teams Up with AMD to Build Next-Gen TPU Hybrid AI Chip: CPU Cores Integrated On-Package

The Strategic Significance of the Google-AMD Partnership Rumors

According to multiple reports, Google is collaborating with AMD to co-design a next-generation Tensor Processing Unit (TPU). This news has attracted significant industry attention because it could mark a major shift in the design paradigm for AI-specific chips (ASICs). The rumored next-gen TPU would adopt a "hybrid AI ASIC" architecture, integrating CPU cores within the chip package, specifically optimized for emerging workloads such as reinforcement learning (RL).

The Evolution of Tensor Processing Units

The Tensor Processing Unit (TPU) is a custom AI chip architecture that Google has been developing since 2016. Unlike general-purpose GPUs, TPUs are specifically optimized for tensor operations in neural networks, excelling particularly at operations like matrix multiplication. TPUs use a Systolic Array architecture, achieving efficient parallel computation through the rhythmic flow of data between processing elements. The first-generation TPU was used for inference only, while training workloads have been supported since the second generation.

For a long time, Google's TPUs have been one of its core competitive advantages in AI infrastructure. From AlphaGo to the TPU clusters that now power Gemini large model training, Google has reduced its complete dependence on NVIDIA GPUs through custom-designed chips. TPUs are offered as a service through Google Cloud Platform, but their full architectural details have not been entirely disclosed. This vertically integrated strategy has helped Google maintain a technological edge in AI infrastructure. The introduction of AMD as a design partner now signals Google's desire to incorporate stronger general-purpose computing capabilities into its next-generation chips.

Strategic Considerations in ASIC Design

An ASIC (Application-Specific Integrated Circuit) is a chip custom-designed for a specific application, in contrast to general-purpose chips like CPUs and GPUs. ASICs trade versatility for peak performance and energy efficiency on specific tasks. In the AI domain, ASICs can optimize data flow for specific neural network architectures and eliminate unnecessary general-purpose features, achieving several times the energy efficiency of GPUs. However, ASICs are expensive to design (potentially tens of millions of dollars), have long development cycles (2–3 years), and struggle to adapt to rapidly evolving AI algorithms. Only companies like Google, with clearly defined, large-scale workloads, can fully leverage the advantages of ASICs.

Why Integrate CPU Cores On-Package

Reinforcement Learning's Unique Demand for Heterogeneous Computing

The core highlight of the rumor is "on-package CPU cores" — integrating CPU cores within the same chip package. The goal of this design is clear: to provide more efficient computing support for reinforcement learning tasks.

Reinforcement learning (RL) differs fundamentally from supervised learning in its computational patterns. Supervised learning primarily involves dense matrix operations well-suited to the parallel architectures of GPUs/TPUs. RL, on the other hand, requires an agent to continuously interact with an environment: at each step, it must execute the policy network (inference) based on the current state, receive rewards, and update the value function. This loop involves a large amount of conditional branching, state transitions, and other control-flow operations. Additionally, RL frequently requires running complex environment simulators (such as physics engines or game engines), which are typically written in serial logic that is CPU-friendly.

When training AlphaGo or Dota 2 AI, for example, the system needs to rapidly switch between neural network inference (where GPUs excel) and Monte Carlo tree search (where CPUs excel). These workloads often involve complex control flow and logical branching — precisely where CPUs are strong and GPUs/TPUs are relatively weak. When RL training needs to frequently switch between "matrix-intensive computation" and "logic control," having the CPU and AI accelerator on separate chips creates significant latency and bandwidth bottlenecks as data travels back and forth between them.

Technical Advantages of Chiplet Package-Level Integration

Placing CPU cores and AI acceleration units within the same package can dramatically shorten the communication path between them, reducing latency and increasing bandwidth. This "near-memory" and "near-compute" design philosophy is closely related to the rise of chiplet technology in recent years.

Chiplet technology involves splitting a single large monolithic chip into multiple smaller dies, which are then integrated using advanced packaging techniques. This approach offers multiple advantages: first, smaller dies have higher manufacturing yields, reducing costs; second, different chiplets can be manufactured using different process nodes (e.g., 5nm for the CPU, 12nm for I/O), achieving an optimal balance of cost and performance; third, it supports heterogeneous integration, flexibly combining different functional modules such as CPUs, GPUs, and memory. Key technologies include high-speed inter-chiplet interconnects (such as AMD's Infinity Fabric and Intel's EMIB/Foveros) and advanced packaging processes (such as 2.5D/3D packaging).

Notably, AMD has deep expertise in chiplet packaging technology. Its EPYC server processors and Instinct series accelerators both employ advanced multi-die packaging designs, integrating up to 17 chiplets in a single package to achieve flexible core configurations and excellent cost-effectiveness. Google's choice to partner with AMD is very likely driven by AMD's leading capabilities in heterogeneous integration and packaging technology.

Shifts in the AI Chip Competitive Landscape

Breaking the Single-Supplier Dependency on NVIDIA

The current AI chip market is highly concentrated, with NVIDIA dominating through its CUDA ecosystem and GPU performance. CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model introduced by NVIDIA in 2006. After nearly 20 years of development, it has formed a powerful ecosystem moat. CUDA encompasses not only programming language extensions but also hundreds of optimized libraries (cuDNN, cuBLAS, TensorRT, etc.), a mature development toolchain, and a vast developer community. Virtually all major AI frameworks (PyTorch, TensorFlow, etc.) are deeply dependent on CUDA. This software lock-in effect means that even when competitors release hardware with comparable performance, developers face enormous migration costs.

Although Google has its own TPUs and has partially circumvented CUDA dependency through its proprietary software stack (the XLA compiler and JAX framework), it still requires external support for general-purpose computing. This collaboration with AMD strengthens Google's autonomy in AI hardware on one hand, while opening a massive custom chip market for AMD on the other.

For AMD, this represents yet another major customer following its custom chip services for Microsoft, Meta, and others. In recent years, AMD has been aggressively expanding its semi-custom business, sharing in the computing dividends of hyperscale data centers by providing design services to cloud providers.

Heterogeneous Computing as the AI Chip Development Trend

This rumor also confirms the broader trend of AI chips evolving from "pure accelerators" to "heterogeneous SoCs." A single type of computing unit can no longer meet increasingly diverse AI workloads. Future AI chips will increasingly tend to tightly integrate different components — CPUs, AI accelerators, high-bandwidth memory — to achieve optimal energy efficiency for specific tasks.

Potential Impact on Future AI Training

If this hybrid chip ultimately comes to fruition, it could have a profound impact on AI training methodologies. As alignment techniques represented by RLHF (Reinforcement Learning from Human Feedback) and reasoning model training become critical stages in large model development, the importance of reinforcement learning is rising rapidly.

RLHF's New Requirements for Computing Architecture

RLHF (Reinforcement Learning from Human Feedback) is currently the core technology for large model alignment. Its workflow is divided into three stages: first, a base model is trained using supervised learning; then, human preference data on model outputs is collected to train a reward model that simulates human preferences; finally, reinforcement learning (typically the PPO algorithm) is used to fine-tune the model so that its outputs maximize the reward model's scores. RLHF enables ChatGPT to refuse inappropriate requests and generate more helpful responses, but it is extremely computationally expensive: it requires simultaneously running the policy model, value model, reward model, and reference model, with memory usage several times that of supervised learning.

The success of OpenAI's o-series, DeepSeek, and other reasoning models all depends on large-scale reinforcement learning training. A chip specifically optimized for RL, integrating both CPU and AI acceleration capabilities, could significantly improve the efficiency of such training, giving Google a hardware-level advantage in the next round of the AI race. This kind of complex multi-model collaborative training is a quintessential scenario demanding new heterogeneous computing capabilities.

A Note of Caution: Still at the Rumor Stage

It's important to emphasize that this information is still at the "according to reports" stage — neither Google nor AMD has officially confirmed any details of the collaboration. Chip design cycles are lengthy, often taking years from concept to mass production, so even if the partnership is real, the product is still some time away from reaching the market.

Nevertheless, the rumor itself reflects an important industry trend: competition in AI chips is shifting from pure computing power benchmarks to architectural innovation targeting specific workloads. Whoever can first design the hardware best suited for the next generation of AI training paradigms may seize the initiative in the future competition for AI infrastructure.