Unified Memory Architecture
The memory architecture used in Apple's M-series chips, which integrates the memory used by the CPU, GPU, and Neural Engine into a single high-bandwidth shared memory pool packaged adjacent to the SoC, eliminating data-transfer latency while creating a strong dependency on high-quality LPDDR memory.
Core Facts
Starting with the M1 chip, Apple uses a Unified Memory Architecture (UMA) where the CPU, GPU, and Neural Engine share the same memory pool.
Apple M系列芯片采用统一内存架构（UMA），CPU和GPU共享同一块高带宽内存池
M1's unified memory architecture allows the CPU and GPU to physically share the same memory, eliminating the need for explicit data copying and requiring only the passing of pointers or virtual addresses.
Timeline (last 90 days)
The diffusers enable_model_cpu_offload() strategy is based on the idea of 'keeping the model on CPU and moving it to GPU only at runtime', but on UMA this strategy may not effectively reduce memory pressure.
On UMA, the so-called 'on CPU' weights are actually sitting in the same unified RAM that the GPU also needs, and moving a tensor from MPS to CPU does not reduce overall memory pressure, only changes the access path and the state of the pages.
Apple Silicon's unified memory architecture can reduce latency and power consumption for small and medium model inference on the basis of single-machine sharing, but once extended to a distributed environment, cross-device communication still has to go over the network, and the bandwidth dividend of unified memory cannot be passed through directly.
On Apple Silicon, the CPU, GPU, and Neural Engine share the same physical memory, eliminating the need to copy data between devices.
The bandwidth of shared system memory is typically lower than that of dedicated GDDR video memory.
Unified memory architecture can significantly reduce the overhead of moving data between different processing units, making it particularly well-suited for scenarios such as running local large models and code-assist tools.
The unified memory architecture eliminates data copy overhead but introduces bandwidth contention issues, and the memory cannot be upgraded.
The Jetson Orin series offers up to 64 GB of unified memory, with the CPU and GPU sharing the same physical memory pool, eliminating the need to copy data between them via the PCIe bus.
The NVIDIA Jetson series adopts a SoC unified memory architecture in which the CPU and GPU share the same physical memory.
M1's unified memory architecture allows the CPU and GPU to physically share the same memory, eliminating the need for explicit data copying and requiring only the passing of pointers or virtual addresses.
3 more timeline events
All Facts (18)
Starting with the M1 chip, Apple uses a Unified Memory Architecture (UMA) where the CPU, GPU, and Neural Engine share the same memory pool.
80%VerifiedApple M系列芯片采用统一内存架构（UMA），CPU和GPU共享同一块高带宽内存池
80%VerifiedM1's unified memory architecture allows the CPU and GPU to physically share the same memory, eliminating the need for explicit data copying and requiring only the passing of pointers or virtual addresses.
75%VerifiedApple Silicon's Neural Engine shares a Unified Memory Architecture with the CPU and GPU, eliminating the bottleneck of data transfer between different memory pools in traditional heterogeneous computing.
75%VerifiedThe AMD Ryzen AI Max+ 395 features a 16-core N5-process CPU paired with 40 RDNA 3.5 compute units and supports up to 128 GB of LPDDR5X unified memory.
65%UnverifiedApple Silicon M-series unified memory architecture allows CPU and GPU to share the same physical memory, eliminating data copy overhead beneficial for LLM inference
90%UnverifiedThe NVIDIA Jetson series adopts a SoC unified memory architecture in which the CPU and GPU share the same physical memory.
70%Partially Verified统一内存架构使CPU、GPU与NPU共享同一块内存池，避免数据在处理单元间反复搬运的开销
65%UnverifiedThe diffusers enable_model_cpu_offload() strategy is based on the idea of 'keeping the model on CPU and moving it to GPU only at runtime', but on UMA this strategy may not effectively reduce memory pressure.
50%UnverifiedOn UMA, the so-called 'on CPU' weights are actually sitting in the same unified RAM that the GPU also needs, and moving a tensor from MPS to CPU does not reduce overall memory pressure, only changes the access path and the state of the pages.
50%UnverifiedApple Silicon's unified memory architecture can reduce latency and power consumption for small and medium model inference on the basis of single-machine sharing, but once extended to a distributed environment, cross-device communication still has to go over the network, and the bandwidth dividend of unified memory cannot be passed through directly.
50%UnverifiedOn Apple Silicon, the CPU, GPU, and Neural Engine share the same physical memory, eliminating the need to copy data between devices.
50%UnverifiedThe bandwidth of shared system memory is typically lower than that of dedicated GDDR video memory.
50%UnverifiedUnified memory architecture can significantly reduce the overhead of moving data between different processing units, making it particularly well-suited for scenarios such as running local large models and code-assist tools.
50%UnverifiedThe unified memory architecture eliminates data copy overhead but introduces bandwidth contention issues, and the memory cannot be upgraded.
50%UnverifiedThe Jetson Orin series offers up to 64 GB of unified memory, with the CPU and GPU sharing the same physical memory pool, eliminating the need to copy data between them via the PCIe bus.
50%UnverifiediPhone最高仅有8GB统一内存，而主流大语言模型参数量超千亿、全精度存储需数百GB内存，使模型压缩成为端侧大模型落地的必要条件
50%Unverified配备统一内存架构的迷你PC能够运行700亿参数（70B）的大模型，而高端独立显卡因显存不足无法完整加载
50%