Unverified50% confidenceFactExact time
The random read performance (IOPS and latency) of the Ngram table on NVMe directly affects first-token latency, not sequential bandwidth or capacity.
1
Sources
50%
Confidence
Long-term
Relevance
11/4/2026
First Seen
Sources
Running 125B MoE on a 12GB GPU? Inside Strata's Three-Tier Cooperative Engine
bilibiliNicksxs11/3/2026
Related Entities
Related Claims
UnverifiedJanas-style streaming loading schemes depend heavily on NVMe, and on SATA drives the random read latency is so high that inference speed drops to an unusable level.77% similarUnverifiedMemory bandwidth determines weight read speed and affects the latency of generating each token, while floating-point compute (FLOPS) determines matrix operation speed and affects the speed of processing input prompts during the prefill stage.74% similarUnverifiedStreaming inference must read and write cache state at every time step; if the state tensors are large or the access pattern is non-contiguous, bandwidth pressure quickly turns into latency.69% similarUnverifiedCoordinated omission refers to how, in a closed-loop load generator, the generator slows its send pace in step when the system under test slows down, causing queueing latency to be systematically underestimated; open-loop scheduling that sends requests on a fixed timetable avoids this problem.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1134855API
curl https://kongchang.com/api/v1/knowledge/claims/1134855MCP
get_claim(id=1134855)