Memory Wall
The phenomenon in which memory access becomes the bottleneck for overall system performance when chip compute capability grows faster than memory bandwidth.
Timeline (last 90 days)
The memory wall refers to the phenomenon where, when chip compute power grows faster than memory bandwidth, actual performance is bottlenecked by memory access speed rather than computing capability.
Memory bandwidth is one of the main bottlenecks for inference latency in large models, where compute units are often waiting for data to load, a phenomenon known as the "memory wall".
Computing power has surged rapidly over the past decade, but the growth rate of memory bandwidth and storage I/O has significantly lagged behind, forming a 'Memory Wall' that has become a critical bottleneck constraining the overall performance of AI systems.
For CNN inference, if intermediate results are frequently written back to memory and then read again, it causes a large number of cache misses, potentially degrading performance by more than 10×.
All Facts (5)
大模型推理是典型的内存带宽密集型任务，GPU计算核心往往处于等待数据的状态
85%UnverifiedThe memory wall refers to the phenomenon where, when chip compute power grows faster than memory bandwidth, actual performance is bottlenecked by memory access speed rather than computing capability.
50%UnverifiedMemory bandwidth is one of the main bottlenecks for inference latency in large models, where compute units are often waiting for data to load, a phenomenon known as the "memory wall".
50%UnverifiedComputing power has surged rapidly over the past decade, but the growth rate of memory bandwidth and storage I/O has significantly lagged behind, forming a 'Memory Wall' that has become a critical bottleneck constraining the overall performance of AI systems.
50%UnverifiedFor CNN inference, if intermediate results are frequently written back to memory and then read again, it causes a large number of cache misses, potentially degrading performance by more than 10×.
50%