Unverified50% confidenceFactExact time
When the KV cache overflows GPU VRAM capacity, performance drops in a step-wise cliff rather than a gradual decline.
1
Sources
50%
Confidence
Long-term
Relevance
10/5/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedKeeping model weights fully resident in GPU memory avoids memory swap-in/swap-out overhead, whereas models that exceed GPU memory capacity require data to be moved between CPU and GPU, causing latency to rise significantly.80% similarUnverifiedIf the GPU is already a bottleneck at the 256GB configuration, doubling the memory to 512GB will not improve GPU compute, and loading larger models will instead result in slower inference.78% similarUnverifiedA GPU that is more powerful but has a higher unit price may actually have a lower cost per token.77% similarUnverifiedGPU 推理速度通常是纯 CPU 的 5-20 倍，当模型太大无法放入显存时部分层会卸载到内存由 CPU 处理（混合运行/partial offload）77% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/977208API
curl https://kongchang.com/api/v1/knowledge/claims/977208MCP
get_claim(id=977208)