Unverified50% confidenceDecision RuleExact time
For inference solutions that rely on host memory offload, the NUMA node where the expert data offloaded to host memory resides should be kept on the same node as the GPU accessing it, otherwise the performance loss will be mistakenly attributed to the offload mechanism itself.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOOM (Out of Memory) is the most commonly encountered failure in local inference: when the VRAM jointly occupied by model weights, KV cache, and concurrent requests exceeds GPU capacity, the CUDA runtime directly terminates the process.70% similarUnverified数据在服务器端进行推理时必然处于明文状态，GPU显存中的激活值、KV缓存以及批处理队列中的请求数据都可能成为侧信道攻击目标64% similarUnverifiedIn the PyTorch DataLoader, num_workers determines how many independent subprocesses are used to read and preprocess data in parallel, and a low value prevents data preparation from overlapping with GPU computation.63% similarUnverifiedProactively invalidating associated memories upon data changes yields a cleaner architecture, but requires tracking the source fingerprint of each memory entry, placing higher demands on system observability and data pipelines.62% similarUnverifiedThe benchmark data comes from a single source shared by a Bilibili UP host, and performance will vary depending on hardware batch, model version, and actual workload.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1044821API
curl https://kongchang.com/api/v1/knowledge/claims/1044821MCP
get_claim(id=1044821)