Unverified50% confidenceFactExact time
Setting vLLM's gpu-memory-utilization to 0.92 under TP=4 causes OOM with the same weights, while setting it to 0.88 runs stably, because CUDA graph capture needs reserved VRAM headroom.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/18/2026
First Seen
Valid until: 1/16/2027
Sources
Related Entities
Related Claims
UnverifiedBy default, vLLM plans 90% of GPU VRAM into the budget via the --gpu-memory-utilization parameter, and only the remainder is used as KV cache.69% similarUnverifiedGPU计算场景中零拷贝可通过CUDA的统一虚拟地址空间（UVA）、页锁定内存（Pinned Memory）和GPUDirect技术实现68% similarUnverifiedSetting a lower num_ctx parameter (such as 2048) in Ollama's Modelfile can limit the context window, reduce KV Cache memory usage, and avoid the sharp performance degradation caused by insufficient memory.68% similarUnverifiedThe GKE managed GPU collector exports only four metrics: duty_cycle, memory_used, memory_total, and request, with no SM occupancy, Tensor Core activity, or power consumption data.67% similarUnverifiedWith Unsloth's 1-bit GGUF quantization, GLM-5.3-Flash can run on a combined 102 GB of RAM and VRAM, with a context length of 1M tokens.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1063642API
curl https://kongchang.com/api/v1/knowledge/claims/1063642MCP
get_claim(id=1063642)