Unverified50% confidenceBenchmarkExact time
vLLM's three optimizations for DeepSeek-V4.1-Flash (SWA bounded replay, MegaAttention and Mega-Gate kernels, operator fusion and CUDA graphs) achieve 5.3x throughput under a 150 TPS constraint, a 1.9x speedup at low concurrency, and about a 70% reduction in TTFT at 100K throughput.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/14/2026
First Seen
Valid until: 1/12/2027
Sources
Related Entities
Related Claims
UnverifiedThe three 4-bit quantization schemes—RTN, GPTQ, and AWQ—have similar compression ratios but progressively increasing precision retention capabilities.72% similarUnverifiedMegatron-LM实验数据表明，NVLink互联下8卡张量并行可达7.5×以上线性加速，而PCIe 3.0 x16环境下7B模型通常仅达2.5-3×，超过4卡后通信开销完全抵消额外算力70% similarUnverifiedThrough request-owned speculative KV, in a DeepSeek-V4-Pro 1P1D, MI355X, concurrency-256 scenario, the average TTFT was reduced from 19.4 seconds to 15.1 seconds.68% similarUnverifiedThe decode unit of DeepSeek V3 uses 144-card EP, amortizing the weight load overhead across large batches and freeing up GPU memory to increase batch size.68% similarUnverifiedRunning MiniMax H3 on an RTX 4070 or below (12 GB or less) may require extreme optimizations such as full model offloading to CPU, reducing resolution, and decreasing frame count.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1026265API
curl https://kongchang.com/api/v1/knowledge/claims/1026265MCP
get_claim(id=1026265)