Unverified50% confidenceFactExact time
Strata's multi-user concurrency uses a parallel slot mechanism, and the slots do not use speculative decoding, causing single-user speed to drop from 144 to 38 tokens/s
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/17/2026
First Seen
Valid until: 1/15/2027
Sources
Related Entities
Related Claims
UnverifiedIn an environment where cgroup limits the CPU quota (about 30 cores) but exposes 256 threads, Strata's default worker pool configuration caused performance to drop to about 30 tokens/s instead of the expected 170 tokens/s76% similarUnverifiedDuring the prefill phase, Strata splits the prompt into chunks of up to 8,000 tokens and streams them over PCIe hidden behind computation, boosting the speed of processing a 32,000-token prompt from approximately 570 tokens/second to nearly 1,300 tokens/second.69% similarUnverifiedWhen the context exceeds 64,000 tokens, Strata keeps most of the KV cache in memory and retains only the hottest 32,000 tokens in VRAM, improving speed from approximately 51 to 63 tokens/second at a 262,000-token context.69% similarUnverifiedOn the same machine and model, the decoding speed difference between low power mode and high power mode is 1.4 to 3.8 times; for example, running Qwen 3.5 35B on Ollama gives 51.9 tokens/s in low power and 107.1 tokens/s in high power.68% similarUnverifiedEven if a small portion of the model overflows into system memory, speed drops dramatically; experiments show the same small model runs at 151 tokens/s in RTX 5090 VRAM but only 8 tokens/s when placed in system memory.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1053018API
curl https://kongchang.com/api/v1/knowledge/claims/1053018MCP
get_claim(id=1053018)