Unverified50% confidenceBenchmarkExact time
Ollama local deployment measured throughput: MacBook M2 16GB approximately 15 tokens/second, RTX 3060 12GB approximately 45, RTX 4090 24GB approximately 70, 8GB laptop approximately 5
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedLocal inference is constrained by the user's hardware: a 7B-parameter quantized model (such as the Q4 version of Llama 3 8B) has a file size of approximately 4–5 GB, can reach 20–40 tokens per second on a Mac with an Apple M-series chip, and may drop to 3–5 tokens per second on a laptop with only an integrated GPU and 8 GB of RAM.78% similarVerifiedAfter 4-bit quantization, Llama 3.3 70B has a model size of approximately 40 GB and can run on a Mac Studio with 64 GB of memory or a single professional GPU with 48 GB of VRAM.72% similarUnverifiedCommunity benchmark comparison data: M2 Ultra approximately 20 tokens/second, M3 Ultra 512 GB at 39.8 tokens/second, a single B200 at 119.7 tokens/second, 8× RTX Pro 6000 at approximately 204 tokens/second, a modified RTX 4090 with 48 GB VRAM on a custom backend at 367 tokens/second, and dual H200 at approximately 375 tokens/second.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1003429API
curl https://kongchang.com/api/v1/knowledge/claims/1003429MCP
get_claim(id=1003429)