Unverified50% confidenceFactExact time
Running Large 4 on a typical laptop with 16GB/32GB of memory generates at about 15 tokens per second.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/30/2026
First Seen
Valid until: 1/28/2027
Sources
Related Entities
Related Claims
UnverifiedTaking a 70B BF16 model as an example, generating one token requires reading approximately 140 GB of weights, and when fully utilizing the H100's 3.35 TB/s memory bandwidth, the per-token latency is at least 42 milliseconds, with a theoretical TPS upper limit of approximately 24 tokens/second.79% similarUnverifiedThe four-quantized version of 2.5-24 is about 13.5G in size and can run on a laptop with 32G of memory; the 6 version is about 18.3G and requires 48G of memory.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1114956API
curl https://kongchang.com/api/v1/knowledge/claims/1114956MCP
get_claim(id=1114956)