Unverified50% confidenceBenchmarkExact time
Running Qwen3-30B-A3B (18.6GB) on a 16GB Tesla T4, using the MoE cache raised decoding speed from the default 22.6 tokens/s to 43.2 tokens/s.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/18/2026
First Seen
Valid until: 11/1/2026
Sources
Related Entities
Related Claims
UnverifiedBased on public data for high power plus MLX or Ollama, the decoding speed of different models on M4 Max is roughly a ladder as follows: Nemotron 3 Nano about 160 tokens/s, Qwen 3.5 35B at 107 to 130 tokens/s, Gemma 4 26B about 75 tokens/s, DeepSeek V4 Flash about 27 tokens/s, and dense 27B about 22 tokens/s.71% similarUnverifiedThe 4-bit quantized weights of Qwen3 27B are only approximately 16.1 GB, allowing it to run on a MacBook Pro/Mac Studio with 32 GB of memory or a consumer-grade GPU with approximately 17 GB of VRAM.71% similarUnverifiedMiniCPM-V 4.7 35B-A3B achieves a decoding speed of 13+ tokens/s on the iPhone 17 Pro Max.71% similarPartially VerifiedLFM2.5-24B-A2B achieves a decoding speed of 184 tokens/s on an Apple M5 Max.70% similarUnverifiedOn a single Mac Studio with 512GB of memory, the 128B Mistral Medium 3.5 (FP8) and the 27B Qwen 3.8 (BF16) occupy about 158GB and 68GB of memory respectively at a 65536-token context.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1060504API
curl https://kongchang.com/api/v1/knowledge/claims/1060504MCP
get_claim(id=1060504)