Unverified50% confidenceFactExact time
Gemma 4 E2B stores only one KV head per layer, and 28 of its 35 layers retain only a 512-token sliding window, with 20 layers reusing the cache of earlier layers.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
单块TPU v5e实测：哪些小模型能与Gemma 4 E2B一战？
Related Entities
Related Claims
UnverifiedEach global layer of Spark X 2.5 stores approximately 4 KB of KV cache per token; across 9 layers this amounts to approximately 37 KB per token, reducing KV cache usage to approximately one-quarter of the approach where every layer records the full text65% similarUnverifiedThe model compresses the cache along three dimensions: 64 query heads share a single latent, every two adjacent tokens in the encoder are compressed into one entry, and only 4 out of 40 layers retain cache while the remaining 36 layers store nothing.65% similarUnverifiedFor the LLaMA-2 70B model, when processing a context of 4096 tokens, the KV Cache requires approximately 20 GB of system memory at FP16 precision.64% similarUnverifiedChoosing Q4_0 for KV cache quantization can compress the cache size to approximately one-quarter of FP16, with far less impact on output quality than model weight quantization.63% similarVerifiedThe VRAM consumption of the KV cache is proportional to context length, number of model layers, and number of attention heads; under an 8K context, a typical 27B model may consume an additional 2–4 GB of VRAM.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1014584API
curl https://kongchang.com/api/v1/knowledge/claims/1014584MCP
get_claim(id=1014584)