Unverified50% confidenceFactExact time
The Qwen3.8-27B checkpoint occupies 28.75 GiB on disk, and after loading in vLLM the weights plus non-torch overhead are about 29.34 GiB, leaving 53.8 GiB on a 96 GB RTX Pro 6000 for the KV cache, equivalent to about 767,317 tokens.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/15/2026
First Seen
Valid until: 1/13/2027
Sources
Related Entities
Related Claims
Verified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB72% similarUnverifiedOn a single RTX PRO 6000, peak VRAM for Qwen3.8-27B-NVFP4 dropped from 72.9 GB to 40.2 GB, and per-step time was reduced by approximately 11%72% similarUnverifiedJanas runs the 48.4GB Qwen3-Next-80B on a 32GB-memory laptop at 37 tok/s; on the same machine and same model llama.cpp reaches only 2.8 tok/s.71% similarUnverifiedThe 3B version of TwIL-LM is approximately 1.78 GiB under Q4_K_M quantization and can run on a CPU or devices with only 4 GB of VRAM.70% similarUnverifiedA used RTX 3090 with 24GB of VRAM costs about 800 USD, can cover 7B to 33B Q4 models, and costs about 15 USD per month in electricity.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1033142API
curl https://kongchang.com/api/v1/knowledge/claims/1033142MCP
get_claim(id=1033142)