Unverified50% confidenceDecision RuleExact time
Inference speed (tokens/second) ≈ memory bandwidth ÷ model size
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedEvery time a model generates a token, the entire model must be read completely from memory, yielding the speed upper bound: memory bandwidth ÷ model size.78% similarUnverifiedThe formula for local AI inference speed is: Token/s ≈ memory bandwidth / amount of data read per token78% similarUnverifiedThe key factors determining what models can be run are memory capacity and memory speed: memory capacity determines model size, and memory speed determines the number of tokens per second.76% similarUnverifiedA 70B-parameter model (140 GB of weights) has a theoretical maximum generation speed of approximately 7 tokens/second at 1 TB/s bandwidth.75% similarVerifiedFor the 4-bit quantized version of a 7B parameter model, at least 8GB of memory and a modern multi-core CPU are typically required for smooth operation, with inference speeds of approximately 10–30 tokens per second75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1032037API
curl https://kongchang.com/api/v1/knowledge/claims/1032037MCP
get_claim(id=1032037)