Unverified50% confidenceFactExact time
llama.cpp processes prompts in batches of 512 tokens by default, while Strata chunks prefill at 6144 tokens, reducing the frequency of experts crossing PCIe.
1
Sources
50%
Confidence
Long-term
Relevance
10/29/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedDuring the prefill phase, Strata splits the prompt into chunks of up to 8,000 tokens and streams them over PCIe hidden behind computation, boosting the speed of processing a 32,000-token prompt from approximately 570 tokens/second to nearly 1,300 tokens/second.73% similarUnverifiedStrata's multi-user concurrency uses a parallel slot mechanism, and the slots do not use speculative decoding, causing single-user speed to drop from 144 to 38 tokens/s69% similarUnverifiedIn an environment where cgroup limits the CPU quota (about 30 cores) but exposes 256 threads, Strata's default worker pool configuration caused performance to drop to about 30 tokens/s instead of the expected 170 tokens/s68% similarUnverifiedWhen the context exceeds 64,000 tokens, Strata keeps most of the KV cache in memory and retains only the hottest 32,000 tokens in VRAM, improving speed from approximately 51 to 63 tokens/second at a 262,000-token context.68% similarUnverifiedReal-world testing with the 284-billion-parameter DeepSeek V4 Flash shows token generation rising from 37 to 53 tokens/s (approximately 1.5x), and Prompt Processing jumping from 483 to 1,485 tokens/s (approximately three times).65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1112093API
curl https://kongchang.com/api/v1/knowledge/claims/1112093MCP
get_claim(id=1112093)