Unverified50% confidenceBenchmarkExact time
When extending the context from 192,000 to 256,000, the expert cache dropped from 16.44 GB to 13.66 GB and KV increased from 4.31 GB to 7 GB, with almost zero speed cost, because KV is fully in VRAM and the expert cache automatically balances.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/5/2026
First Seen
Valid until: 1/3/2027
Sources
Related Entities
Related Claims
VerifiedThe VRAM consumption of the KV cache is proportional to context length, number of model layers, and number of attention heads; under an 8K context, a typical 27B model may consume an additional 2–4 GB of VRAM.70% similarUnverifiedThis configuration achieves three optimizations—262K ultra-long context, KV Cache Q4 quantization, and MTP speculative sampling—pushing 24GB VRAM utilization to 99%69% similarVerifiedThe size of the KV cache grows linearly with context length; the KV cache for Qwen 2.5 32B at a 64K context may consume several additional gigabytes of memory.69% similarVerifiedQuantized KV caches (such as storing KV vectors in FP8 or INT4) can compress the cache size to 1/2 or even 1/4 of the original.69% similarUnverifiedThe Q8 main model occupies 162 GB, preventing the drafter from fitting into unified memory, rendering DSpark acceleration ineffective; after moving three layers of MoE experts back to the CPU, output only improved from 22.87 to 23.49 tokens/second (2.7% faster), while prefill dropped from 95.38 to 76.65.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/978746API
curl https://kongchang.com/api/v1/knowledge/claims/978746MCP
get_claim(id=978746)