Unverified50% confidenceFactExact time
For a 27B-class model, even if the weights are compressed to around 5GB, the KV cache under long contexts can still occupy several GB, becoming the real bottleneck for phone memory.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/17/2026
First Seen
Valid until: 1/15/2027
Sources
Related Entities
Related Claims
VerifiedThe VRAM consumption of the KV cache is proportional to context length, number of model layers, and number of attention heads; under an 8K context, a typical 27B model may consume an additional 2–4 GB of VRAM.77% similarUnverifiedThe 2-bit quantized DeepSeek 763B model takes up about 175GB of memory, can run on a 256GB machine and leaves room for the KV cache.77% similarUnverifiedKV cache is voluminous, potentially reaching several GB for a single request under long contexts, posing challenges for disk read/write and capacity management.77% similarVerifiedModel weights cannot fill up VRAM entirely, as space must be reserved for the KV cache and inference processes; the KV cache required for a 128K context can reach as high as several GB or even exceed the model weights themselves.77% similarUnverifiedIn Agent workloads, the context expands from 60,000 to 400,000 tokens, and the KV cache of a single session may occupy tens of GB of GPU memory; although requests are waiting during gaps between tool calls, the cache cannot be released, further reducing the effective batch size.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1054989API
curl https://kongchang.com/api/v1/knowledge/claims/1054989MCP
get_claim(id=1054989)