Unverified50% confidenceBenchmarkExact time
On an RTX 3060, the developer's hand-written Triton fused W4A16 GEMV kernel targeting batch-1 decoding runs at about 25.5 tok/s, matching the roughly 25.7 tok/s of the bitsandbytes NF4 quantization scheme.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/12/2026
First Seen
Valid until: 10/26/2026
Sources
Related Entities
Related Claims
VerifiedIn benchmark testing, Qwen3 27B achieved a generation speed of approximately 40-something tokens per second on 4 modified 2080 Ti cards without NVLink enabled.79% similarUnverified开发者公布在RTX 3050 Laptop（4GB）上微调Llama-3.1-8B的实测结果为训练速度119.6 tok/s，峰值显存3.32 GB76% similarUnverifiedThe RTX 4090 has a theoretical FP16 compute performance of 330 TFLOPS, and the actual Decode speed for a 7B FP16 model is typically 50–60 tokens/s.75% similarUnverifiedThe typical generation speed of an NVIDIA RTX 4090 running a similarly scaled Q4 quantized model is in the range of 40–60 TPS74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1006832API
curl https://kongchang.com/api/v1/knowledge/claims/1006832MCP
get_claim(id=1006832)