Unverified50% confidenceBenchmarkExact time
On a single RTX Pro 6000, using MTP speculative decoding (K=3) reduces single-stream per-token latency from 22 ms to 13 ms, increases the number of concurrent requests per card from 32 to 64, and raises aggregate output from 999.9 tokens/sec to 1954.0 tokens/sec.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/15/2026
First Seen
Valid until: 10/29/2026
Sources
Related Entities
Related Claims
UnverifiedOn an RTX Pro 6000 (96 GB VRAM), the official FP8 version combined with built-in MTP speculative decoding can reach approximately 120 tokens/second, with quality nearly indistinguishable from 16-bit.75% similarUnverifiedA speed of 4–5 tokens/second is too slow for practical use on an 8GB GPU; a 16GB GPU can improve this to 12–14 tokens/second, entering a practically usable range.72% similarUnverified将MTP草稿深度从默认4提升至6或8时，Qwen3.8-27B的输出速度分别降至53.9和52.4 tokens/秒，低于默认的55.1 tokens/秒。72% similarUnverifiedWith a 5070 12 GB GPU paired with 64 GB of memory, speed can reach 93 tokens/s in short chat scenarios and 74 tokens/s for 128K long contexts.71% similarUnverifiedIn terms of prompt processing speed, the RTX 4090 achieves approximately 3,000 tokens/s and the M5 Max approximately 800 tokens/s, giving NVIDIA a clear advantage in long-document and codebase input scenarios.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1033164API
curl https://kongchang.com/api/v1/knowledge/claims/1033164MCP
get_claim(id=1033164)