Unverified50% confidenceFactExact time
When config.json is available, ModelFit uses the GQA formula to precisely calculate the KV cache: 2 × layers × kv_heads × head_dim × ctx × 2 bytes; otherwise it uses the heuristic rule of "+20% with a minimum of 1.5GB" and clearly marks this on the page.
1
Sources
50%
Confidence
Long-term
Relevance
10/17/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFor a 70B model using GQA (n_layers=80, n_kv_heads=8, head_dim=128, FP16 cache), the KV Cache size per token is about 320 KB.75% similarVerified4-bit quantization in GGUF format can compress model weights from 32-bit floating point to 4-bit integers, reducing model size by approximately 8 times.74% similarUnverifiedAccurately estimating a model's VRAM usage requires considering multiple dimensions including KV Cache, inference framework runtime overhead (typically an additional 0.5–2 GB), and activation values.70% similarVerifiedModel weights cannot fill up VRAM entirely, as space must be reserved for the KV cache and inference processes; the KV cache required for a 128K context can reach as high as several GB or even exceed the model weights themselves.70% similarUnverifiedModelFit uses a default context length of 8K and notes that the KV cache grows linearly with context.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1054739API
curl https://kongchang.com/api/v1/knowledge/claims/1054739MCP
get_claim(id=1054739)