Running AI Inside DRAM: A Processing-in-Memory Experiment That Breaks DDR4 Timing Rules

Introduction: When Memory Itself Becomes the Compute Unit

In the traditional von Neumann architecture, data must be shuttled back and forth between the CPU and memory — a bottleneck known as the "Memory Wall," which has become one of the biggest obstacles to performance and energy efficiency in modern computing systems. Since its introduction in 1945, the von Neumann architecture has dominated computer design, with its defining characteristic being the separation of memory and processor, sharing a single bus for both instructions and data. Driven by Moore's Law, processor performance improved at roughly 60% per year, while DRAM access latency improved at only about 7% per year. This widening gap was formally dubbed the "Memory Wall" in the 1990s. By the 2020s, a single off-chip DRAM access consumes roughly 200–1000× the energy of a floating-point operation, and in large language model inference, memory bandwidth — not compute — is often the true bottleneck. This is especially pronounced in AI inference scenarios, where the massive movement of weight data consumes far more energy than the actual computation itself.

Recently, a Hacker News project titled "Running PrismML's Bonsai inside DRAM by breaking DDR4 timing rules" caught the attention of the tech community. The project proposes a bold and counterintuitive idea: by deliberately violating DDR4 memory timing rules, the DRAM chip itself can be made to perform computation, enabling PrismML's lightweight Bonsai machine learning model to run directly inside memory.

This is a quintessential hacker-style exploration of Processing-in-Memory (PIM), and its technical approach deserves a deep dive.

What Are Processing-in-Memory and the Bonsai Model?

Processing-in-Memory: The Core Idea Behind Breaking the Memory Wall

The fundamental premise of PIM is simple: if moving data is so expensive, why not bring computation to where the data lives? DRAM, as an extremely dense storage medium, inherently possesses a massively parallel row/column structure. The basic DRAM storage cell consists of one transistor and one capacitor (the 1T1C structure), with data stored as electrical charge on the capacitor. The entire storage array is organized into rows (Row/Wordline) and columns (Column/Bitline), with each bank containing tens of thousands of rows and each row containing thousands of cells. During a read operation, the row decoder activates a wordline, and the charge from all cells in that row simultaneously flows onto their respective bitlines. Sense amplifiers then amplify the tiny voltage differentials into full logic levels. This massively parallel physical structure is naturally suited for bulk operations and provides the physical foundation for processing-in-memory.

Researchers have discovered that by cleverly manipulating DRAM's electrical and timing characteristics, certain logic operations — such as bitwise AND, OR, and XOR — can be performed "on the fly" during read or refresh cycles.

In recent years, several academic projects have explored similar ideas. Notable examples include "Ambit" and "ComputeDRAM," both of which attempt to leverage commodity DRAM to trigger RowClone and bulk bitwise computation by violating standard timing specifications. Ambit (published at IEEE Micro 2017) is a landmark work from Onur Mutlu's group at Carnegie Mellon University. It achieves Majority Gate (MAJ) operations by simultaneously activating three rows, and combined with NOT operations, can construct complete Boolean logic. ComputeDRAM (2019) further demonstrated that on unmodified commodity DRAM chips, AND and OR operations can be performed across different rows by reducing the tRAS and tRP parameters. Subsequent works like DRISA and Fulcrum explored integrating more complex computational logic into sense amplifiers. Together, these studies form a comprehensive academic landscape of "leveraging DRAM physics for computation."

Bonsai: An AI Model Designed for Extreme Resource Constraints

PrismML's Bonsai is a lightweight tree-based model specifically designed for extremely resource-constrained devices such as IoT sensors and microcontrollers. Bonsai belongs to the EdgeML project family, jointly developed by Microsoft Research and IIT, alongside models like ProtoNN and FastGRNN. Its core idea is to apply sparse projections to traditional decision trees, compressing model parameters down to the 2–16KB range while maintaining classification accuracy close to that of random forests or small neural networks.

Its defining characteristics are an extremely small model footprint (compressible to just a few KB) and an inference process that relies primarily on simple comparisons and additions. During inference, Bonsai mainly performs vector dot products and threshold comparisons — operations that can be decomposed into multiply-accumulate and sign-determination steps, which can be further simplified into bitwise and shift operations. The regularity and simplicity of this computational pattern make it an ideal workload for PIM systems. This "small but beautiful" nature makes it the perfect candidate for running inside DRAM — because the "compute capability" DRAM can offer is extremely primitive, suitable only for simple, highly parallel operations.

Core Technique: Deliberate DDR4 Timing Violations to Enable Computation

Why DDR4 Timing Rules Exist

The DDR4 specification defines strict timing parameters such as tRCD (Row-to-Column Address Delay), tRAS (Row Active Time), and tRP (Precharge Time). These parameters ensure that under normal operating conditions, the charge in DRAM capacitors can be reliably read and refreshed, thereby guaranteeing data integrity.

The DDR4 standard is defined by JEDEC (Joint Electron Device Engineering Council), and timing parameters are set with ample safety margins (Guard Bands). Take tRCD as an example: JEDEC specifies a typical value of 13.75ns, but research has shown that many DRAM chips can complete row activation in as little as 7–10ns. tRAS (minimum row active time, typically 33ns) ensures the sense amplifier has sufficient time to complete charge recovery. tRP (precharge time, typically 13.75ns) ensures bitline voltages return to their equilibrium state. The existence of these margins is precisely what makes "timing violation" techniques possible — chips don't fail immediately when operated out of spec but instead exhibit observable and partially predictable degraded behavior.

The Controlled "Side Effects" of Breaking Timing Rules

The key insight of this project is that when these timing intervals are deliberately shortened, DRAM exhibits predictable, non-standard behavior. For example:

• Triggering another operation before a row has fully activated can cause multiple rows to be activated simultaneously, producing a charge-sharing effect.
• The result of charge sharing can be interpreted as the logical operation result across multiple memory cells (e.g., majority voting / majority gate).

From a physical standpoint, when two rows are activated nearly simultaneously, the charge from both storage cells is injected onto the same bitline. The final bitline voltage depends on the superposition of the two cells' charges: if both cells store "1" (high charge), the voltage offset is large, and the sense amplifier determines "1"; if one stores "1" and the other "0," the offset falls in an intermediate zone, and the sense amplifier's determination depends on its threshold setting — essentially implementing AND or OR logic. Activating three rows simultaneously implements majority voting (MAJ3). By orchestrating different activation sequences and preset data patterns, Boolean operations can be systematically constructed. This is the core mechanism for transforming analog electrical phenomena into digital computational primitives.

By carefully choreographing these "violation" operation sequences, the developers effectively transform DRAM's analog electrical characteristics into primitive computational building blocks. Mapping Bonsai's inference logic onto these primitives enables true "in-memory AI inference."

Technical Significance and Real-World Challenges

The Potential Value of Processing-in-Memory

The significance of this experiment is reflected in several dimensions:

1. Enormous energy efficiency potential: If computation can be performed where the data resides, the energy consumed by data movement is drastically reduced — particularly attractive for edge AI devices.
2. Hardware reuse lowers cost: No dedicated AI accelerator chip is needed; computation is performed directly on commodity DRAM, reducing hardware costs.
3. Academic research value: It reveals the vast untapped potential that exists "outside the spec" of commercial hardware, providing empirical references for future PIM chip design.

Real-World Limitations Today

However, this approach also faces significant challenges:

• Reliability concerns: Violating timing rules is essentially exploiting "undefined behavior." Results are highly dependent on the specific chip, temperature, voltage, and even individual unit variation, making cross-device consistency difficult to guarantee.
• Limited computational precision: The operational primitives DRAM can provide are extremely limited, suitable only for minimalist models like Bonsai — they cannot support complex neural network inference.
• Data corruption risk: Out-of-spec operations may corrupt other data in memory, requiring strict isolation of computation regions.

Industry Trends: From Lab to Commercialization

It's worth noting that processing-in-memory is not merely an academic concept — the industry has already made substantial moves. Samsung launched its HBM-PIM product with integrated PIM functionality in 2021, embedding compute logic directly into the HBM memory stack. SK Hynix's AiM (Accelerator-in-Memory) chip targets embedding lookup scenarios for recommendation systems. Startups like Mythic use flash storage arrays for analog matrix multiplication, while Untether AI places compute units adjacent to SRAM. These commercial efforts share the same philosophical lineage as the DDR4 hack discussed here but take a more engineered, production-ready approach. Gartner predicts that by 2026, more than 10% of edge AI chips will integrate some form of near-memory or in-memory computing functionality.

Conclusion: Hacker Spirit and Pushing Hardware Boundaries

Although "Running Bonsai inside DRAM" is currently a research-oriented proof of concept, it embodies a valuable spirit of exploration — treating hardware specifications not as absolute boundaries, but actively seeking to uncover hidden capabilities that hardware possesses under extreme conditions.

As AI applications push further toward edge and endpoint devices, energy efficiency and cost become increasingly critical considerations. The processing-in-memory pathway may gradually evolve from academic experiment to practical deployment. This project may be small, but it's a fascinating piece of the puzzle leading toward a future vision of "compute everywhere." For technology practitioners focused on low-level hardware and AI efficiency, it offers an alternative perspective well worth contemplating.