Unverified50% confidenceBenchmarkExact time
In BF16 configuration (two 48GB GPUs), decoding speed ranges from 25.1 tok/s at 1K to 23.9 tok/s at 128K, with almost no degradation.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/5/2026
First Seen
Valid until: 1/3/2027
Sources
Related Entities
Related Claims
UnverifiedIn real-world testing on an RTX 3060 (6GB VRAM) with 32GB RAM running the pre-quantized Krea2 Turbo, 1024×1024 resolution with 8-step Euler sampling achieved a speed of 1.78 s/it and a total elapsed time of 17.64 seconds.76% similarUnverifiedTaking a 70B BF16 model as an example, generating one token requires reading approximately 140 GB of weights, and when fully utilizing the H100's 3.35 TB/s memory bandwidth, the per-token latency is at least 42 milliseconds, with a theoretical TPS upper limit of approximately 24 tokens/second.76% similarUnverifiedThis solution allows a GPU with 16GB of VRAM to run a 27B-class model with a 192K context and vision capability, but the speed ceiling is constrained by hardware bandwidth.76% similarUnverifiedIn a prefill-only test running gpt-oss-120b on 4× B200s, DWDP4 achieves a 1.92× speedup over DEP4 under the MNT 32K/ISL 32K configuration and a 1.54× throughput improvement at saturation (506K vs. 329K tok/s).76% similarUnverifiedCompressing the context from 128K to 32K may quadruple the number of concurrent users a single GPU can serve, representing a trade-off between throughput and context length.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/977115API
curl https://kongchang.com/api/v1/knowledge/claims/977115MCP
get_claim(id=977115)