Unverified50% confidenceBenchmarkExact time
At batch 1, the RTX 3090's inference throughput is about 1.03~1.09 times that of the L40S (based on Qwen2.5 1.5B/7B models, measured on vLLM 0.27.1).
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOn an RTX 3060, the developer's hand-written Triton fused W4A16 GEMV kernel targeting batch-1 decoding runs at about 25.5 tok/s, matching the roughly 25.7 tok/s of the bitsandbytes NF4 quantization scheme.70% similarUnverifiedThis benchmark ran the local Qwen model on a single RTX 3090 GPU70% similarVerifiedQLoRA introduces 4-bit quantization technology, making it possible to fine-tune models with 65 billion parameters on a single RTX 3090/4090 GPU with 24 GB of VRAM.68% similarUnverifiedRTX 4090相比RTX 3060在推理场景下的实际吞吐量提升约3-4倍67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1061229API
curl https://kongchang.com/api/v1/knowledge/claims/1061229MCP
get_claim(id=1061229)