Unverified50% confidenceBenchmarkExact time
JetSpec++ reaches a generation speed of up to 1,361 tokens/second for a single request on NVIDIA GB200.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/15/2026
First Seen
Valid until: 1/13/2027
Sources
Related Entities
Related Claims
UnverifiedThe NVIDIA A100 has a memory bandwidth of 2 TB/s; loading a 280 GB FP32 model takes approximately 140 ms, meaning weight loading alone limits generation to a maximum of approximately 7 tokens per second.72% similarUnverifiedLFM2.5-24B-A2B achieves throughput of over 21,000 tokens/second on a single NVIDIA B300 private cloud accelerator.72% similarUnverifiedIn a 16-core environment, Delta Tokenizer encodes a 4-million-character request 8.2 times faster than serial encoding and 1.6 to 2.1 times faster than LoPT.67% similarUnverifiedIn the test, model loading took only a few seconds, GPU usage dropped from 23GB to about 17GB, the first test prompt processing took 225ms, generation speed was stable at 33.5 tok/s, with a total output of 1500 tokens.67% similarUnverifiedUsing NVIDIA B300 chips in a 1-megawatt data center can achieve a total throughput of approximately 450,000 tokens/s, which at 340 tokens/s per user amounts to approximately 1,324 concurrent active users.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1038127API
curl https://kongchang.com/api/v1/knowledge/claims/1038127MCP
get_claim(id=1038127)