Unverified50% confidenceBenchmarkExact time
NVIDIA engineers, by writing a brand-new dedicated fused decoding kernel upstream for VLLM, achieved a throughput of 423 tokens per second per user on the 280-billion-parameter Kimi K3 model, a 1.68x speedup on equivalent hardware.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/18/2026
First Seen
Valid until: 1/16/2027
Sources
OpenCode Suffers Distillation Attack; DeepSeek Open-Sources Flash 1017 — AI News Roundup
bilibiliAl早知道10/18/2026
Related Entities
Related Claims
UnverifiedThe HD Noise Canceling Processor QN1 used in the Sony XM4 performs more than 700 operations per second.69% similarUnverifiedOn a desktop CPU of similar performance, the generation speed of a 7B Q4 quantized model is roughly between a few tokens and a dozen-plus tokens per second.68% similarUnverifiedVKORE increases single-user inference response speed by 3.8x on the same NPU hardware.67% similarUnverifiedMeta provides two dedicated draft models for speculative decoding: DFlash V2 is designed for SGLang and is reportedly achieving a 3.7x throughput improvement on AMD MI355X hardware, accepting an average of 6.4 draft tokens per step; DSpark is designed for vLLM and proposes up to seven tokens per pass.67% similarVerifiedThe M4 chip's Neural Engine can perform 38 trillion operations per second (38 TOPS)67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1059776API
curl https://kongchang.com/api/v1/knowledge/claims/1059776MCP
get_claim(id=1059776)