Unverified50% confidenceBenchmarkExact time
The 35B model can achieve 45 to 60 tokens per second on a 4090-class GPU.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
VerifiedA 35B model requires approximately 70GB of VRAM at FP16 precision, far exceeding the RTX 5090's 32GB capacity, so single-GPU deployment requires quantization.76% similarUnverified测试者用双4090（48GB显存）工作站运行20.7B稠密模型，每秒生成27.6个Token，仅比矮星略快，但参数量小10倍以上71% similarUnverifiedTaking a 70B BF16 model as an example, generating one token requires reading approximately 140 GB of weights, and when fully utilizing the H100's 3.35 TB/s memory bandwidth, the per-token latency is at least 42 milliseconds, with a theoretical TPS upper limit of approximately 24 tokens/second.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1001696API
curl https://kongchang.com/api/v1/knowledge/claims/1001696MCP
get_claim(id=1001696)