Unverified50% confidenceFactExact time
In large model inference, GPU floating-point compute doubles every two years, while the growth of HBM/DRAM bandwidth lags noticeably behind.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGPU memory bandwidth growth has long lagged behind compute growth, causing the actual time of operations like attention to be determined by data movement between HBM and on-chip SRAM rather than pure computation volume.73% similarUnverifiedIf the GPU is already a bottleneck at the 256GB configuration, doubling the memory to 512GB will not improve GPU compute, and loading larger models will instead result in slower inference.72% similarUnverifiedLarge model inference relies on GPU clusters, and the supply of high-end GPUs is constrained by multiple factors such as production capacity, export controls, and data center power capacity, with expansion cycles measured in quarters or even years.68% similarUnverifiedCompared to traditional contiguous allocation, PagedAttention can increase the number of concurrent requests on the same GPU by 2–4 times.68% similarUnverifiedWhen the KV cache overflows GPU VRAM capacity, performance drops in a step-wise cliff rather than a gradual decline.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1117260API
curl https://kongchang.com/api/v1/knowledge/claims/1117260MCP
get_claim(id=1117260)