Deep Dive into OpenAI's Custom AI Chip: Software Stack, Architecture, and the Threat to NVIDIA

OpenAI's custom AI chip project — internally codenamed with a pepper-themed naming system including Jalapeño, Habanero, and others — has recently had its technical details brought to light, with a clearer picture emerging of everything from the software toolchain to rack-level hardware architecture. According to analyses of related technical articles, this system demonstrates serious ambitions to compete head-on with NVIDIA's Blackwell across software development efficiency, kernel programming languages, and scale-out network design. This article breaks down its core technical approach and analyzes the potential impact on NVIDIA's competitive moat.

Software Stack: A Codex-Driven Kernel Development Pipeline

OpenAI polishes its chip kernels with the same precision used to write Chrome-grade code. Each kernel is hand-tuned, with some reaching roughly 3,000 lines of code, complete with correctness verification and custom Sanitizer tooling. Early kernel development relied on manual intervention, but as internal versions of Codex scaled up, the process has steadily shifted toward greater automation.

What's particularly noteworthy is the software pipeline's real-world performance: before benchmarking against DeepSeek^Ad, OpenAI had not yet implemented certain critical kernel types internally — yet Codex was able to rapidly generate fully functional, high-performance kernel code without any intervention from the kernel engineering team. This clearly demonstrates that its AI-assisted programming capabilities are already capable of handling the heavy lifting of low-level chip software. OpenAI also plans to extend its internal Codex capabilities to enterprise customers, forming an externally-facing product line.

Gluon and Linear Layouts: A Custom Kernel Programming Language

OpenAI programs its chip using a language called Gluon — a custom-built kernel programming language built on top of Triton. It preserves Triton's SPMD (Single Program Multiple Data) programming model while exposing lower-level programming abstractions. For NVIDIA GPUs, it provides APIs that map to PTX instructions, including MMA instructions, TMA instructions, and the M-Barrier mechanism.

Gluon's most distinctive abstraction is the Layout. A Layout defines the mapping between hardware resources (e.g., the 5th accumulator of the 9th warp) and tensor elements (e.g., row 6, column 7). Gluon's Layout abstraction is built on Linear Layouts — an algebraic type system for layouts invented by OpenAI that formally defines the concept of a layout mathematically and provides tools for manipulating Layouts, enabling provably correct layout transformations and optimal in-kernel memory access patterns.

On the programming model side, each Gluon program maps to a persistent thread, indicating that the chip is designed for persistent kernel programming: each program executes across multiple compute units, with work distribution handled by the programmer rather than a hardware scheduler. The chip also provides data prefetching and out-of-order execution units; users can programmatically wait on prefetched data, with data locked via semaphores.

Triton is a GPU programming language previously open-sourced by OpenAI, designed to let researchers write high-performance kernels without needing deep CUDA expertise. It uses the SPMD model, where programmers describe computation at block granularity and the compiler handles thread-level parallelism automatically. Gluon extends Triton downward, directly exposing PTX (NVIDIA Parallel Thread Execution instruction set) level primitives — MMA (matrix multiply-accumulate) instructions for Tensor Core computation, TMA (Tensor Memory Accelerator) instructions for asynchronous data movement, and M-Barrier, a lightweight synchronization primitive introduced in CUDA 12. These low-level interfaces allow Gluon to precisely control data pipelining and compute overlap, achieving performance approaching hand-written assembly while remaining more readable and composable than raw PTX. Linear Layout, its core algebraic tool, formalizes the mapping from tensor elements to hardware registers as linear functions, enabling static verification of layout correctness and eliminating the large class of runtime layout alignment errors common in traditional kernel development.

Remarkable Development Speed: Rapid Iteration with Simulator Support

Cross-time comparisons show that this chip's development has progressed at a remarkable pace. In less than two weeks, throughput in certain interactive scenarios more than doubled. Within just eight days, the team scaled parallel configurations from TP8 to TP32, from a single system to a full rack-level configuration, and ran repeated validation passes on large models.

To validate performance before committing to actual hardware, OpenAI developed a simulator accurate to within 5% of measured hardware results. The team also demonstrated running an internal model (codenamed 5.3 Codex Spark) in the Codex Command environment, with each output token taking approximately 1.2 milliseconds. Even more interesting was a demo running directly on the chip — including DOOM running at 36 FPS, single-precision floating-point fluid dynamics simulations, and interactive drag visualizations — used to concretely demonstrate the chip's general-purpose computing capabilities.

The PD Disaggregation Trade-off: Why They Skipped Prefill-Decode Separation

One surprisingly counterintuitive design choice: OpenAI chose not to implement a Prefill-Decode (PD) disaggregated architecture on these chips. This is notable because NVIDIA and AMD GPUs both show significant performance gains from PD disaggregation, even in homogeneous hardware environments.

OpenAI's reasoning centers on the dynamic nature of production environments. In fixed-load scenarios, PD disaggregation is genuinely attractive — the two stages place very different demands on hardware resources, and assigning them to separately optimized resource pools improves efficiency. But production traffic doesn't maintain fixed ratios: input/output sequence lengths, concurrency, cache hit rates, speculative decoding acceptance rates, and latency targets all vary dynamically. Once hardware is partitioned into fixed resource pools, excess prefill demand leaves decode chips idle, and vice versa — operators must continuously predict and balance a system whose ideal ratio is always shifting.

In a unified architecture, certain resources may be underutilized during specific phases, but each device can still respond to the next request. In a disaggregated system, a chip may sit idle simply because it belongs to the wrong resource pool — local utilization looks higher, but global utilization is actually worse. Additionally, PD disaggregation breaks data locality: the large KV caches generated by prefill nodes must be transferred over the network to decode nodes. As input sequences grow longer and KV caches grow larger, bandwidth consumption, synchronization overhead, and latency all increase, introducing new failure domains. The same constraint applies to speculative decoding — separating the draft model from the verifier turns a tight decode loop into a distributed protocol, where the additional communication overhead may negate the latency savings from the draft mechanism.

That said, this choice isn't without trade-offs. In scenarios with sufficiently large, stable, and predictable demand, a modular PD-disaggregated architecture may still win out.

The prefill stage processes all tokens in the input prompt at once in a highly compute-intensive operation — a classic compute-bound bottleneck. The decode stage autoregressively generates one token at a time; each step processes only one new token but must repeatedly read the full KV cache, making it memory bandwidth-bound. The dramatically different compute-to-bandwidth ratios between the two stages gave rise to the idea of assigning them to heterogeneous or separate resource pools — prefill nodes can be configured with more compute units, while decode nodes can prioritize high-bandwidth memory (HBM). Google TPU's "disaggregated serving" and NVIDIA's Dynamo inference framework are both exploring this direction. OpenAI's decision to forgo PD disaggregation is not a rejection of its theoretical benefits, but rather a judgment that in their highly dynamic production traffic environment, the scheduling rigidity of static resource partitioning and the overhead of cross-node KV cache transfer would offset or even reverse those benefits — a trade-off consistent with their operational philosophy of prioritizing global utilization over local utilization.

Rack-Level Architecture: A Hardware Matrix of Pepper-Named Components

The entire system adopts a pepper-naming scheme spanning Japanese to Indian varieties, mild to scorching. At the rack unit level, the system consists of CPU host racks and ASIC racks. Each host rack contains 16 host CPU trays, with each tray paired to an ASIC tray. Each host is configured with two CPUs, 1.5TB of memory, two M.2 SSDs, and a 400G (2×200G) front-end network interface, connected to the ASIC tray via 8 external PCIe DAC cables.

Each ASIC rack consists of 16 ASIC trays and 8 scale-out switch trays. Each ASIC tray contains 8 XPUs, giving each rack a total of 128 XPUs. The connectivity approach is broadly similar to NVIDIA's rack scale-out architecture. The system is divided into two domains: a local domain within the rack containing 128 ASICs, and a global domain supporting up to 16 racks with a combined 2,048 ASICs.

On the power side, the Sidecar host rack draws approximately 50kW (31kW in production), the ASIC rack draws 130kW, and the dual-rack system totals around 160kW — roughly equivalent to the power consumption of a dual-chassis GB300 rack.

Scale-Out Network: A 2,048-XPU Cluster Powered by Tomahawk 6

OpenAI can connect up to 2,048 XPUs within a single scale-out network. The network consists of two domains: the local domain connects all 128 XPUs within a rack via backplane; the global domain connects 2,048 XPUs across 16 racks via a hybrid copper and optical interconnect.

Each rack contains 8 switch module trays: the middle 6 serve the local domain, each equipped with one 102.4T Tomahawk 6 switch chip; the top and bottom two serve the global domain, each composed of two Tomahawk 6 chips for a per-tray switching capacity of 204.8T. The local domain provides 4.8TB/s unidirectional bandwidth per XPU, with full connectivity to 6 Tomahawk 6 chips. Each XPU requires 48 differential pairs for connectivity, and each rack contains a total of 6,144 differential pairs of passive copper DAC cables.

The global domain uses a purely optical interconnect architecture, routing associated paths through optical circuit switches installed in each rack. Each XPU's global unidirectional bandwidth is approximately 1.6TB/s. Signals travel via backplane to the global switch tray, pass through 1.6T front-panel transceivers into the optical switch, and exit the rack. Since the scale-out network accounts for only 10% of total system cost, this flexibility provides valuable headroom for future ultra-large models supporting 10 to 20 trillion parameters or context windows of 2 to 4 million tokens.

Tomahawk 6 is a Broadcom Ethernet switch chip targeting hyperscale data centers, with a single-chip switching capacity of 102.4Tbps supporting 800G and higher-rate ports — one of the highest bandwidth-density commercial switch chips currently available. OpenAI's choice of Ethernet-based Tomahawk 6 over InfiniBand (Mellanox's primary interconnect technology, now under NVIDIA) means its scale-out network is completely decoupled at the protocol layer from NVIDIA's NVLink/InfiniBand ecosystem. The advantages of Ethernet include an open ecosystem, diverse equipment sourcing, cost control, and native interoperability with external IP networks; the trade-off is that reliable transport and congestion control must be implemented in software (similar to RoCEv2) to compensate for native Ethernet's limitations with latency-sensitive and loss-sensitive workloads. Using backplane passive copper (DAC) in the local domain reduces latency and power consumption, while switching to optical interconnects in the global domain enables longer-distance connectivity. This tiered hybrid design has precedent in HPC environments and balances cost against performance effectively.

A Twist of Fate: NVIDIA GPUs Helping Build Their Own Rival

There's a deliciously ironic fact here: OpenAI's GPT-series models currently run on NVIDIA GPUs — and are being used to design a chip that poses a genuine threat to NVIDIA's CUDA moat. In other words, NVIDIA's own GPUs are actively accelerating the birth of their potential successor in real time.

Jensen Huang has already taken notice of recent developments. He previously made public statements suggesting OpenAI has no need to build its own chips — a remarkably restrained comment, but the underlying threat is obvious. On the deployment front, OpenAI is working with emerging cloud providers to collect reliability data and optimize deployment timelines from dock to rack. The first production tokens are imminent, with the next milestone targeting 100 megawatt scale.

AI is moving at breakneck speed, and capital always flows toward profit. OpenAI's progress on custom silicon is, in many ways, the latest illustration of this logic: as the software toolchain, programming language, and hardware architecture converge into a closed loop, NVIDIA will no longer face competition on a single dimension — but rather a challenge from a complete, vertically integrated hardware-software system.