Unverified50% confidenceFactExact time
Only the cold portion of the KV cache is offloaded; weights, indices, and GDN states remain resident on the GPU, while the full history is stored in host memory and fetched back on demand.
1
Sources
50%
Confidence
Long-term
Relevance
9/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe disk caching mechanism persists already-computed KV Cache to storage media; when a subsequent request's prefix hits the cache, it can be reused directly, skipping redundant forward propagation.69% similarUnverifiedSGLang introduces a weight caching daemon that retains weights for each GPU, allowing the engine to recover from cache rather than storage on restart to shorten fault recovery time.68% similarUnverifiedPersistent queues write messages to disk or distributed storage to ensure that queued requests are not lost even after a system restart, whereas in-memory queues lose data when the process crashes67% similarVerifiedThe GPU memory hierarchy from fastest to slowest is: registers, L1 cache/shared memory, L2 cache, global memory (VRAM).66% similarUnverifiedTraditional KV cache implementations pre-allocate contiguous GPU memory blocks of maximum length, resulting in an average of 60%–80% of pre-allocated memory being wasted.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/923187API
curl https://kongchang.com/api/v1/knowledge/claims/923187MCP
get_claim(id=923187)