Unverified50% confidenceBenchmarkExact time
On Strix Halo, the model's cold-start prefill reached several hundred tokens per second and output held steady at just over 40 TPS; on the CUDA workstation, prefill reached several thousand tokens per second and output sustained between 100 and 2100 TPS.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/29/2026
First Seen
Valid until: 1/27/2027
Sources
Related Entities
Related Claims
UnverifiedDuring the prefill phase, Strata splits the prompt into chunks of up to 8,000 tokens and streams them over PCIe hidden behind computation, boosting the speed of processing a 32,000-token prompt from approximately 570 tokens/second to nearly 1,300 tokens/second.70% similarUnverifiedWhen the context exceeds 64,000 tokens, Strata keeps most of the KV cache in memory and retains only the hottest 32,000 tokens in VRAM, improving speed from approximately 51 to 63 tokens/second at a 262,000-token context.69% similarUnverifiedThe Gemma open-source model achieved 2.85x end-to-end inference acceleration on A100 via vLLM, with throughput increasing from about 37,900 tokens/second to over 108,000 tokens/second, while maintaining 100% token consistency and zero quality degradation.66% similarUnverifiedIn an environment where cgroup limits the CPU quota (about 30 cores) but exposes 256 threads, Strata's default worker pool configuration caused performance to drop to about 30 tokens/s instead of the expected 170 tokens/s66% similarUnverifiedWhen running 4-bit DeepSeek R1 on the M3 Ultra, a 1,000-token context generates approximately 19 tokens per second using 392GB, dropping to approximately 15 tokens per second using 410GB at a 16,000-token context.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1112037API
curl https://kongchang.com/api/v1/knowledge/claims/1112037MCP
get_claim(id=1112037)