Unverified50% confidenceFactExact time
Swapping KV cache in and out depends on the interconnect bandwidth between host memory and GPU VRAM; taking PCIe 4.0 x16 as an example, the one-way effective bandwidth is on the order of about 25 GB/s, while the HBM bandwidth of a single data center GPU can reach 2–3 TB/s, a difference of about two orders of magnitude.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGPU HBM bandwidth is typically on the order of 2–3 TB/s (3.35 TB/s for H100), while the effective bandwidth of Host DRAM connected via PCIe 5.0 is only about 64 GB/s—a difference of nearly 50x.79% similarUnverifiedGPU interconnect bandwidth within data centers is typically on the order of 100–400 Gbps (via InfiniBand HDR/NDR or NVSwitch), while wide-area network bandwidth between data centers may be only 1–10 Gbps.76% similarVerifiedOn traditional x86 platforms, the bandwidth ceiling of the PCIe bus connecting a discrete GPU to the motherboard memory is approximately 64GB/s.75% similarUnverifiedGPU video memory (VRAM) bandwidth is typically on the order of 500–1,000 GB/s, whereas DDR5 system memory bandwidth is only approximately 50–80 GB/s, a difference of more than 10x.75% similarVerifiedThe 3080 Ti has a memory bandwidth of 912 GB/s, far exceeding the 16–32 GB/s PCIe bus bandwidth between CPU and GPU.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1017345API
curl https://kongchang.com/api/v1/knowledge/claims/1017345MCP
get_claim(id=1017345)