Analog Compute-in-Memory: How Mythic Uses the Laws of Physics to Break Through AI Chip Energy Efficiency Bottlenecks

Introduction: The Energy Efficiency Dilemma of AI Computing

With the explosion of large models and edge AI applications, traditional computing architectures are hitting an insurmountable "memory wall." Under the von Neumann architecture, data must be shuttled back and forth between the processor and memory, and the energy consumed by this process often far exceeds that of the actual computation itself. According to multiple industry estimates, data movement can account for over 60% of total power consumption in deep learning inference tasks.

The von Neumann architecture was born in 1945, with its core characteristic being that programs and data share the same memory space, with the CPU fetching instructions and data from memory via a bus. This architecture has supported the entire computer industry's development over the past several decades, but as processor speeds have improved far beyond memory bandwidth growth (processor performance growing ~60% annually while DRAM bandwidth grows only ~10% per year), an increasingly severe performance gap has formed between the two. In deep learning workloads, this contradiction is further amplified: a large Transformer model may have billions of parameters, and each inference requires moving these parameters from DRAM to compute units, with data movement energy consumption reaching 200 times that of the computation itself. This is why even though GPUs possess formidable computing power, actual energy efficiency remains limited by the physical bottleneck of data movement.

It is against this backdrop that Mythic's Analog Compute-in-Memory architecture has sparked lively discussion in the HackerNews community (garnering 47 upvotes and 23 comments). This technological approach attempts to fundamentally restructure the relationship between computation and storage, moving multiply-accumulate operations directly inside the storage units themselves.

What is Analog Compute-in-Memory

The Core Idea Behind Breaking the "Memory Wall"

Traditional digital chips read data from memory to compute units, complete operations, and then write results back. The core concept of Compute-in-Memory (CIM) is to make the memory itself capable of computation, thereby eliminating the massive energy consumption and latency caused by data movement.

What makes Mythic unique is its choice of the analog computing path. It uses Flash Memory cells to simultaneously store neural network weights and directly execute analog-domain matrix multiplication within the storage array. Flash memory was originally designed as a non-volatile data storage device, with its basic unit being the Floating Gate Transistor. By injecting different amounts of charge into the floating gate, the transistor's threshold voltage can be changed, enabling multi-bit storage. Mythic cleverly exploits this characteristic: the amount of charge stored in the floating gate determines the transistor's conduction current at a specific gate voltage, and this conductance value can be programmed to represent a neural network weight. Unlike traditional digital storage that only distinguishes between 0 and 1, analog compute-in-memory requires flash cells to provide multi-level, continuous conductance states, placing higher demands on device programming precision and state retention capabilities. NOR Flash, due to its support for random access and better current consistency, is typically more suitable for compute-in-memory than NAND Flash.

Computing with the Laws of Physics

The elegance of Mythic's architecture lies in its clever application of two fundamental physical laws:

• Ohm's Law: Current equals voltage divided by resistance. When input signals are applied as voltages to memory cells, by controlling the flash cell's conductance (equivalent to weight), the output current naturally equals "voltage × conductance" — completing a multiplication operation.
• Kirchhoff's Current Law: Currents in parallel branches automatically sum. When currents from multiple cells in the same column converge, the accumulation portion of the multiply-accumulate (MAC) operation is naturally achieved.

This means a complete vector-matrix multiplication can be completed in one clock cycle, physically in parallel, without individual digital operations. MAC (Multiply-Accumulate) operations are the most fundamental and computationally intensive operations in deep learning inference. In a typical fully connected layer or convolutional layer, the input vector needs to be dot-multiplied with a weight matrix — this is essentially a collection of massive MAC operations. Taking a 1024×1024 matrix multiplication as an example, approximately 1 million MAC operations are needed. In modern large language models (GPT-scale), each inference may involve trillions of MAC operations. Traditional digital chips must execute these operations sequentially or with limited parallelism, while the revolutionary aspect of analog compute-in-memory is this: a storage array with 1024 rows can simultaneously complete 1024 multiplications and automatically accumulate them in a single clock cycle, achieving truly massive parallel computation. For neural network inference workloads dominated by matrix multiplication, this is an extremely well-matched architectural design.

Energy Efficiency Advantages and Technical Highlights of Analog Compute-in-Memory

Weights Reside in Storage, Completely Eliminating Data Movement

Mythic stores neural network weights directly in analog flash cells — once written, weights remain stationary. During inference, only input activations flow through the array, avoiding the enormous overhead of repeatedly loading weights from DRAM. This is especially valuable for models with massive parameter counts — when weights don't move, energy consumption naturally drops dramatically.

Mixed-Signal Design Combining Digital and Analog

Although core computation occurs in the analog domain, Mythic's architecture is not a purely analog system. It uses DACs (Digital-to-Analog Converters) at the array's input to convert digital activation values into analog voltages, and ADCs (Analog-to-Digital Converters) at the output to convert accumulated currents back to digital signals. This Mixed-Signal design allows the chip to enjoy the energy efficiency advantages of analog computation while seamlessly interfacing with external digital systems.

DACs and ADCs are the bridges connecting the digital world to the analog computing domain. ADC design is one of the key bottlenecks of the entire system: high-precision ADCs (12-bit and above) are large in area, high in power consumption, and slow; low-precision ADCs, while efficient, introduce additional quantization noise. The industry typically uses SAR (Successive Approximation Register) ADCs or Sigma-Delta ADCs to balance precision and efficiency. Research estimates suggest that in some compute-in-memory designs, ADCs may occupy 30-50% of total chip area and 40-60% of total power consumption, meaning ADC optimization directly determines whether compute-in-memory architectures can truly achieve their theoretical energy efficiency advantages.

Precise Positioning for Edge AI Inference

Mythic's architecture primarily targets edge inference scenarios, such as smart cameras, AR/VR devices, industrial vision, and other applications highly sensitive to power consumption and cost. In these scenarios, energy efficiency ratio is often more critical than absolute computing power.

Edge AI inference refers to running AI models directly on end devices (rather than in cloud data centers) for inference. These scenarios have several distinct characteristics: first, strict power constraints — battery-powered devices may only allow power budgets of a few hundred milliwatts to a few watts; second, latency sensitivity — real-time video analysis, autonomous driving decisions, and similar applications require millisecond-level responses and cannot tolerate round-trip network latency to the cloud; third, cost sensitivity — consumer and industrial devices have strict chip cost limitations. Additionally, edge devices face privacy protection requirements (data stays on-device) and the need for independent operation in offline environments. These constraints align remarkably well with the advantages of analog compute-in-memory — low power, low latency, high energy efficiency ratio — which is also the strategic logic behind Mythic's decision to target the edge market first rather than data centers.

Technical Challenges and Community Discussion

Inherent Challenges of Analog Computing

In the HackerNews community discussion, many engineers pointed out the real-world challenges facing analog compute-in-memory:

• Precision and Noise Issues: Analog computation is inherently affected by noise, temperature drift, and device mismatch, making it difficult to achieve the deterministic precision of digital computation. While neural networks have some tolerance for low precision, ensuring consistency across batches and across chips remains a challenge.
• ADC/DAC Conversion Overhead: The converters themselves consume significant area and power. Some argue that if conversion overhead is too large, it could negate the energy savings from compute-in-memory — this is a key metric for evaluating the true benefits of such architectures.
• Limited Flexibility for Weight Updates: Flash memory has slow write speeds and limited endurance (typical NOR Flash supports approximately 100,000-1,000,000 program/erase cycles), making this architecture more suitable for inference tasks with fixed weights rather than training scenarios requiring frequent updates.

Compiler and Toolchain Maturity

Analog architectures place extremely high demands on software toolchains. Developers need to "map" standard neural network models onto analog arrays, handling a series of issues including quantization, calibration, and error compensation. This involves complex engineering challenges such as quantizing floating-point weights to the limited precision levels supported by flash cells, modeling and compensating for systematic errors introduced by analog computation, and considering the impact of array physical layout on computational accuracy during compilation. Toolchain maturity often determines whether such novel architectures can be truly adopted by industry.

Industry Outlook and Development Directions for Compute-in-Memory

Analog compute-in-memory is not an exploration unique to Mythic. In recent years, both academia and industry have conducted extensive research on compute-in-memory solutions based on various devices including Flash, ReRAM, and Phase-Change Memory (PCM).

Beyond flash, ReRAM (Resistive RAM) changes resistance by forming or breaking conductive filaments in metal oxide thin films. It offers advantages of fast write speeds (nanosecond-level), low power consumption, and scalability to a few nanometers, but faces challenges of high device-to-device variability and limited endurance. PCM (Phase-Change Memory) uses phase transitions between crystalline and amorphous states in chalcogenide materials to achieve different resistance states, offering strong multi-level storage capability and good retention characteristics — IBM and Intel have invested heavily in this direction. MRAM (Magnetoresistive RAM) uses resistance changes in magnetic tunnel junctions, offering the fastest speed but relatively limited analog precision. Each device type has its unique precision-speed-endurance tradeoffs, and no single solution currently dominates across all metrics.

This field is widely regarded as one of the important breakthroughs for improving AI energy efficiency in the post-Moore's Law era. Moore's Law drove exponential growth in computing performance over the past half-century, but as process nodes approach physical limits (quantum tunneling effects at atomic scales, thermal limits of power density, etc.), relying solely on process shrinkage to improve performance is no longer sustainable. Post-Moore computing development is pursuing multiple parallel paths: first, breaking through area limitations via 3D stacking and advanced packaging (such as Chiplets); second, developing specialized accelerators (such as TPUs and NPUs) to improve efficiency for specific workloads; third, exploring entirely new computing paradigms including analog computing, quantum computing, photonic computing, and neuromorphic computing. Compute-in-memory belongs to the third category and is one of the directions closest to commercialization, because it can leverage mature semiconductor manufacturing processes without depending on entirely new physical platforms like quantum computing — giving it a significant advantage in engineering feasibility.

However, between technology demonstrations and large-scale commercialization lie multiple hurdles including reliability, yield, toolchains, and ecosystem development. Mythic's own development has experienced several ups and downs, indirectly reflecting the difficulty of commercializing cutting-edge architectures.

For practitioners following AI hardware, Mythic's analog compute-in-memory provides a highly instructive case study: it demonstrates the possibility of stepping outside the digital computing paradigm and using physical laws to directly perform computation. Regardless of whether this approach ultimately becomes mainstream, the "computation as storage" concept it explores will continue to influence the design philosophy of next-generation AI chips.

Conclusion

Mythic's analog compute-in-memory architecture is fundamentally a rethinking of the computing paradigm itself. By storing weights in flash cells and leveraging Ohm's Law and Kirchhoff's Current Law to complete matrix multiplication in the analog domain, it has the potential to achieve orders-of-magnitude energy efficiency improvements in edge AI inference scenarios.

But as community discussions reveal, precision control in analog computing, ADC/DAC overhead, and software toolchain maturity remain barriers that must be overcome for this approach to achieve large-scale deployment. In an era where AI computing demands continue to surge while energy efficiency increasingly becomes a bottleneck, such architectural explorations undoubtedly deserve close attention from the entire industry.