Unverified50% confidenceFactExact time
In the default per_loop mode, Dory's KV cache multiplies with the number of loops: about 14 GiB for a 256K context at 2 loops and about 24 GiB at 4 loops; switching to last_loop mode reduces this to about 9 GiB (256K).
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedKV缓存的规律是：上下文长度翻倍缓存大致翻倍，并发满上下文请求翻倍缓存再翻倍76% similarUnverifiedThe KV cache size can be expressed with the formula cache = 2 (K and V) × layers × KV heads × head dim × tokens × 2 bytes, where 2 bytes corresponds to FP16, doubling when using FP32, and halving when using INT8 quantization.70% similarVerifiedThe VRAM consumption of the KV cache is proportional to context length, number of model layers, and number of attention heads; under an 8K context, a typical 27B model may consume an additional 2–4 GB of VRAM.69% similarVerifiedThe size of the KV cache grows linearly with context length; the KV cache for Qwen 2.5 32B at a 64K context may consume several additional gigabytes of memory.67% similarVerifiedThe VRAM usage of KV Cache grows linearly with sequence length; for a 70-billion-parameter model using GQA, the KV Cache usage can reach tens of GB when processing 100,000 tokens.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1063743API
curl https://kongchang.com/api/v1/knowledge/claims/1063743MCP
get_claim(id=1063743)