Unverified50% confidenceFactExact time
The official Qwen3-8B configuration has 36 layers, 8 KV heads, head dimension 128, and sliding window attention disabled; with 8192 tokens cached, 2 bytes per element, and a single sequence, the KV cache is about 1.125 GiB.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/18/2026
First Seen
Valid until: 1/16/2027
Sources
Related Entities
Related Claims
UnverifiedEach of Qwen3-27B's 16 full-attention layers has 4 KV heads and a head dimension of 256, occupying 64 KB per token at 16-bit precision.79% similarUnverifiedLlama 3 8B has 32 layers, 8 KV heads (GQA architecture with 32 query heads), a head dimension of 128, and a KV cache of approximately 1.07 GB for a single sequence under FP16 with an 8192 context length.71% similarUnverified使用IQ4_XS配合-fa 1 -ctk q8_0 -ctv q8_0对KV缓存进行q8_0量化后，可加载完整32k上下文，常驻内存约16.6GB71% similarUnverifiedQwen3 27B features 64 layers, a hidden dimension of 5120, and a native context length of 262,144 tokens.70% similarUnverifiedQwen-2引入了GQA、支持128K token的上下文窗口和基于字节级BPE的分词器（词表约15万），参数规模从0.5B到72B70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1057305API
curl https://kongchang.com/api/v1/knowledge/claims/1057305MCP
get_claim(id=1057305)