Unverified50% confidenceFactExact time
Betelgeuse keeps the approximately 3.2GB dense layers + GDN + QSA + LM head in VRAM, and the remaining VRAM holds about 5,000 expert slots, distributed as: 240 slots per layer for early layers 0–4, 92 slots for middle layers, and 60 slots for tail layers.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/31/2026
First Seen
Valid until: 1/29/2027
Sources
Running a 68GB Large Model on a 12GB GPU: Betelgeuse Engine Benchmarked at 22 tok/s
redditr/LocalLLaMA10/31/2026
Related Entities
Related Claims
UnverifiedFor a 70B model using GQA (n_layers=80, n_kv_heads=8, head_dim=128, FP16 cache), the KV Cache size per token is about 320 KB.69% similarUnverifiedvLLM's PagedAttention splits the KV cache into fixed-size physical blocks (typically 16 tokens), improving GPU memory utilization from below 40% to close to 90%.68% similarUnverifiedFor ModelFit on Kimi K3 (unsloth repository), the UD-Q1_0 quantization has a measured file size of 466.4 GB and requires 561.1 GB of VRAM/RAM at an 8K context; UD-Q4_K_XL has a measured file size of 1.51 TB and requires about 1.6 TB; F16 is about 3 TB.68% similarUnverifiedWith 8 concurrent slots, VRAM remaining under the default configuration was less than 400MiB, and the driver began paging; adding --vram-reserve-mib 1536 resolves this issue67% similarUnverifiedThe formula for total VRAM requirements for MoE models in production is: VRAM_total = (VRAM_weights + VRAM_KV(C) + VRAM_CUDA) × 1.06, which includes at least a 6% HBM3e fragmentation buffer.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1120766API
curl https://kongchang.com/api/v1/knowledge/claims/1120766MCP
get_claim(id=1120766)