Unverified50% confidenceBenchmarkExact time
The Glacier coordinator splits Qwen2-57B-A14B (57B parameters, 14B activated per token, 8/64 MoE) across 4 nodes at 7 layers per node for a total of 28 layers, with a disk footprint of about 25.6GB (Q3_K_M quantization).
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedEach of Qwen3-27B's 16 full-attention layers has 4 KV heads and a head dimension of 256, occupying 64 KB per token at 16-bit precision.68% similarVerified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB65% similarUnverifiedQwen3.8-Max-0902 uses a Mixture-of-Experts (MoE) architecture with 2.4T parameters and supports a context length of 128K.64% similarUnverifiedFor a typical 7B model (32 layers, 32 heads, 128 dimensions) processing a 2048-token sequence at FP16 precision with a batch size of 8, the KV Cache consumes approximately 4 GB of VRAM.64% similarUnverifiedQwen3-235B-A22B adopts a MoE architecture, activating only 22B out of 235B total parameters at a time.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1010454API
curl https://kongchang.com/api/v1/knowledge/claims/1010454MCP
get_claim(id=1010454)