Unverified50% confidenceFactExact time
UP主使用FP8精度部署千问3.8 27B在48GB显卡上，跑4个并发request占用约46GB
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/18/2026
First Seen
Valid until: 11/16/2026
Sources
Related Claims
Unverified35B模型在FP16精度下需要约70GB显存，远超RTX 5090的32GB容量，因此单卡部署需量化75% similarVerified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB72% similarUnverified双卡 24GB 张量并行配置在 f16 精度下上下文支持从 134,049 增至 262,144 token72% similarUnverifiedFP8量化可将7B模型权重从约14GB（FP16）压缩至约7GB72% similarUnverifiedNVIDIA A100的显存带宽为2TB/s，加载280GB的FP32模型需要约140ms，仅权重加载就决定了每秒最多生成约7个token71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/768404API
curl https://kongchang.com/api/v1/knowledge/claims/768404MCP
get_claim(id=768404)