Unverified50% confidenceFactExact time
On an H100, the GPU completes one computation step far faster than data loading, and a low num_workers causes the dataloader to be unable to feed data in time, forcing the GPU to idle and wait, i.e. the data starvation problem.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
Unverified传统GPU如NVIDIA A100/H100在推理阶段面临内存带宽瓶颈，模型权重需要在每次生成Token时从显存反复读取77% similarUnverifiedKeeping model weights fully resident in GPU memory avoids memory swap-in/swap-out overhead, whereas models that exceed GPU memory capacity require data to be moved between CPU and GPU, causing latency to rise significantly.75% similarUnverifiedAn extremely large prompt may completely occupy the GPU for several minutes during the prefill phase, causing queue congestion when multiple users run full-context tasks simultaneously.72% similarUnverifiedWhen the KV cache overflows GPU VRAM capacity, performance drops in a step-wise cliff rather than a gradual decline.71% similarUnverified启用 AMP 会缩短 GPU 消化一个 batch 的时间，从而放大 DataLoader 供给不足造成的瓶颈71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1022866API
curl https://kongchang.com/api/v1/knowledge/claims/1022866MCP
get_claim(id=1022866)