Unverified60% confidenceFactExact time
In a 170k token scenario, the attention score matrix requires approximately 54 GB of VRAM in BF16 precision (170,000 × 170,000 × 2 bytes).
2
Sources
60%
Confidence
Long-term
Relevance
8/10/2026
First Seen
Sources
Related Claims
UnverifiedEach of Qwen3-27B's 16 full-attention layers has 4 KV heads and a head dimension of 256, occupying 64 KB per token at 16-bit precision.76% similarVerifiedAt a sequence length of 4096, the N×N attention matrix alone consumes approximately 64 MB of memory in FP16 precision.76% similarUnverified128K上下文窗口的注意力矩阵规模是32K窗口的16倍73% similarUnverified128K长度的序列所需的注意力计算量是4K长度序列的1024倍73% similarVerifiedTaking a 70B parameter model with FP16 precision and 32 attention head layers as an example, the KV Cache for a single 200K-length request requires approximately 80–120 GB of VRAM, far exceeding the 80 GB VRAM limit of a single A100 GPU.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/727004API
curl https://kongchang.com/api/v1/knowledge/claims/727004MCP
get_claim(id=727004)