Unverified50% confidenceSolutionExact time
Every time a model generates a token, the entire model must be read completely from memory, yielding the speed upper bound: memory bandwidth ÷ model size.
1
Sources
50%
Confidence
Long-term
Relevance
10/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe key factors determining what models can be run are memory capacity and memory speed: memory capacity determines model size, and memory speed determines the number of tokens per second.79% similarUnverifiedThe upper limit of system memory capacity directly determines the size of model a user can run without sacrificing additional precision.75% similarUnverifiedEach time a token is generated, the Mac must read the entire model from memory, making memory bandwidth the fundamental upper limit on inference speed.72% similarUnverifiedThe formula for local AI inference speed is: Token/s ≈ memory bandwidth / amount of data read per token72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/973164API
curl https://kongchang.com/api/v1/knowledge/claims/973164MCP
get_claim(id=973164)