Verified65% confidenceFactExact time
CPU offload places some Transformer layers in system main memory to be computed by the CPU, with data transferred over the PCIe bus (PCIe 4.0 x16 theoretical bandwidth is approximately 32 GB/s), while GPU memory bandwidth is typically in the range of hundreds to over a thousand GB/s, making the bandwidth gap the fundamental cause of the performance bottleneck.
3
Sources
65%
Confidence
Long-term
Relevance
8/12/2026
First Seen
Sources
Related Claims
UnverifiedPCIe Gen4 x16 bandwidth is approximately 64GB/s, which often becomes a hard bottleneck in memory offloading scenarios.80% similarUnverifiedTo ensure inference speed within 16 GB of VRAM, the core principle is to avoid having weights offloaded to memory, because there is nearly a 10x difference between GPU VRAM bandwidth and CPU memory bandwidth.77% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/735391API
curl https://kongchang.com/api/v1/knowledge/claims/735391MCP
get_claim(id=735391)