Unverified50% confidenceFactExact time
A ~25 GB Q3 model running entirely within the RTX 4090's VRAM (bandwidth ~1 TB/s) theoretically achieves ~40 tokens per second, whereas running it in DDR5 memory (~80 GB/s) drops throughput to 1–3 tokens per second.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/9/2026
First Seen
Valid until: 11/7/2026
Sources
本地跑DeepSeek前沿模型：24GB显卡也能玩转
Related Claims
UnverifiedA single RTX 4090 with 24 GB of VRAM can run the 4-bit quantized version of Qwen3-32B smoothly.78% similarUnverifiedThe RTX 4090 has 24 GB of VRAM, and after loading the 27B Q4 model, there is typically enough space remaining to support a context of 30,000–40,000 tokens.77% similarUnverifiedFor the RTX 4090 (24GB VRAM), WhichLLM may recommend the Q5 quantized version of Qwen3.6 27B, with an estimated inference speed of approximately 27 tokens per second.76% similarUnverifiedA model with 70 billion parameters (70B) can run on a single RTX 4090 (24GB VRAM) after quantization.75% similarUnverifiedQwen2.5-Coder-32B running with Q4 quantization on an RTX 4090 can achieve an inference speed of 40–60 tokens per second.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/716946API
curl https://kongchang.com/api/v1/knowledge/claims/716946MCP
get_claim(id=716946)