Unverified50% confidenceFactExact time
NoteBank's persistent storage is approximately 1/2600 of a full multi-head attention KV cache of equivalent width, depth, and precision (74.5 MiB versus 192 GiB in the paper's Figure 1).
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedThe VRAM consumption of the KV cache is proportional to context length, number of model layers, and number of attention heads; under an 8K context, a typical 27B model may consume an additional 2–4 GB of VRAM.75% similarUnverifiedEngram dedicates one quarter of the entire checkpoint's capacity to n-gram memory, reaching a scale of 197B parameters.73% similarUnverifiedStoring the KV cache at 8-bit precision can save memory and support larger context windows; the author's daily use involves context lengths of 130,000 to 260,000.73% similarUnverifiedFor the LLaMA-2 70B model, when processing a context of 4096 tokens, the KV Cache requires approximately 20 GB of system memory at FP16 precision.72% similarVerifiedFor a typical 7B model with 32 layers and 32 attention heads, storing the KV Cache for each token requires approximately 0.5 MB of VRAM, so generating 1,000 tokens requires an additional ~500 MB of VRAM71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1040229API
curl https://kongchang.com/api/v1/knowledge/claims/1040229MCP
get_claim(id=1040229)