Running Local LLMs on M4 Max: The Three Switches Behind 42 vs 130 tokens/s

The same M4 Max, the same model, one run at 42 tokens/s, another at 130 tokens/s. The difference isn't the chip, nor the model, but three settings most people never notice: runtime, power mode, and model architecture. A video reposted on Bilibili from overseas aggregated five public benchmarks, labeled the source of every number on screen, and drew conclusions you can apply directly.

The Reality Behind the Three Numbers: Speed Is Waiting Time

A coding agent can generate 2000 tokens in a single response. At 42 tokens/s, that's about 48 seconds of waiting; at 130 tokens/s, it compresses to about 15 seconds. At 50 responses per day, a slow configuration devours an entire afternoon.

The video set a basic threshold: all models compared were released within the past 12 months and verified on Hugging Face, including Qwen 3.5/3.6/3.8, Google Gemma 4, NVIDIA Nemotron 3 Nano, and DeepSeek^Ad V4 Flash.

Switch One: Runtime Determines a 1.5x to 3x Gap

In March 2026, Ante Kapetovic ran Qwen 3.5 35B (4-bit weights) on a 128GB M4 Max: MLX hit 130.2 tokens/s, LLAMA.cpp 72.4, and Ollama landed between 41 and 48. In his words, MLX is two to three times faster than Ollama and about one and a half times faster than LLAMA.cpp, while the model itself occupies only about 20GB of memory.

But the plot quickly reversed. Twelve days later, Ollama removed the old Apple engine, and Ollama 0.19, released March 30, switched to the MLX backend on Apple Silicon. In Ollama's own tests, decode speed jumped from 58 tokens/s to 112. Note the fine print: this release came out as a preview, supports only one model, requires Macs with more than 32GB of memory, and the new tests used a different NVFP4 format. By June, Ollama said the engine was another 20% faster, with NVFP4 about 20% faster than the old Q4 format.

How MLX, LLAMA.cpp, and Ollama Relate

These three are not competitors at the same layer. LLAMA.cpp is the low-level C/C++ inference framework started by Georgi Gerganov, supporting backends like CPU, CUDA, and Metal, compatible with the GGUF quantization format—nearly every local inference tool is built on top of it. MLX is an array computation framework from Apple's own machine learning team, designed specifically for the unified memory architecture of M-series chips, calling Metal and the Neural Engine directly, so it often runs faster on models well optimized for Apple Silicon. Ollama is not an inference engine but a packaging and scheduling layer: it pulls models, manages context, exposes APIs, and can swap the underlying engine. This is exactly why the article notes that after "Ollama removed the old Apple engine and switched to the MLX backend, speed changed dramatically"—the same Ollama, a different engine, completely different performance. The tokens/s number a user sees is actually determined by which engine Ollama calls plus that engine's version, not by Ollama itself.

Switch Two: Power Mode, the Variable Everyone Ignores

QPark ran the same prompts and models on two 128GB M4 Macs in low-power and high-power modes. On the same machine, speeds differed by 1.4x to 3.8x. Ollama running Qwen 3.5 35B: 51.9 tokens/s in low power, 107.1 in high power. The dense Qwen 3.6 27B doubled from 11 to 22. DeepSeek V4 Flash rose from about 9 to 27, nearly triple. Short-response scenarios were even more extreme: 28.5 tokens/s jumped straight to 109.6, a 3.84x gap, with power settings as the only variable.

QPark himself flagged the limitation: each case was run only once, no averaging. But that's enough to show that if previous benchmark videos were shot in low-power mode, that Mac may have been underestimated by three times. His advice is simple and direct: check the power mode first.

What Apple Silicon's Low Power Mode Actually Does

macOS Low Power Mode isn't simply slower fans or power-saving standby—it actively throttles the SoC. When enabled, the system lowers CPU and GPU power ceilings, limits sustained turbo capability, and on some models reduces memory bandwidth scheduling priority and slows P-core wake frequency. For LLM inference—a workload highly dependent on memory bandwidth and sustained matrix operations—these limits show up directly in decoded tokens per second. More importantly, the switch is tucked away in System Settings, and many benchmarkers don't realize it's on, which is why the same machine can differ by three times across videos. High Power Mode removes these limits, letting the chip run at full turbo within thermal constraints. Note that High Power Mode triggers throttling faster under prolonged load, so single short tests and long-task real-world results may diverge—which is also why benchmark data needs to note sustained duration.

Switch Three: The Structural Chasm Between MoE and Dense Models

A Mixture-of-Experts (MoE) model wakes only a small portion of about 3 billion parameters per token. Under high-power operation, a 35B MoE model reaches 78 to 107 tokens/s, while a dense 27B gets only 22. NVIDIA's Nemotron 3 Nano also activates about 3B parameters per token; John Rocky's Apple Silicon LLM benchmark recorded MLX at 159.7 tokens/s and LLAMA.cpp at 86.2, but the two runs used inconsistent quantization.

These numbers come from different people, dates, and formats—not a clean controlled experiment—but the pattern holds in every source: MoE takes off, dense crawls.

Why MoE Runs Faster at the Same Size

Mixture of Experts splits a model's feedforward layers into many "expert" subnetworks, and for each token, a routing network selects only a few experts to participate in computation. So a 35B-total-parameter MoE model may only activate about 3B parameters per token, yet the model still carries 35B of storage and knowledge capacity. This is the source of "large footprint, little compute": VRAM or unified memory must hold all the weights, but actual computation is close to a small model, so decode speed approaches a small dense model while answer quality approaches a large one. Dense models have no such path—every parameter is computed for every token; 27B really computes 27B, so speed is naturally pinned low. Understanding this makes the "MoE takes off, dense crawls" ladder in the article clear: speed differences come from activated parameter count, while memory footprint comes from total parameter count—the two must be viewed separately.

Counterexample: MLX Isn't Always Twice as Fast

Most videos overlook one thing. Ash's LLM Picker project collected 92 public Mac measurements and found that on most models MLX leads by only 1x to 1.2x, with the big gaps concentrated in the Qwen 3.5 and 3.6 series. For example, Gemma 4 26B on M3 Max: MLX 65.0, LLAMA.cpp 63.9—essentially tied.

On the flip side, Ollama sometimes wins. Jordan Ilchif tested Qwen 3.8 27B on a 32GB M4 MacBook Air: Ollama's NVFP4 build hit 17.2 tokens/s, pure MLX 6.6, LLAMA.cpp 4.4. But he cautioned that the formats differ. In a fair 4-bit comparison, MLX with D-Flash 2 speculative decoding won, 16.6 vs Ollama Q4's 5.7; on 35B MoE, MLX 32.8, Ollama 29.9. His conclusion: MLX's lead has shrunk to about 10%, and the claim that "MLX is always twice Ollama" is outdated.

Why Quantization Format and Speculative Decoding Affect Comparability

Quantization compresses model weights from high precision (like FP16) into lower-bit representations to save memory and boost speed; common ones include Q4, 4-bit, and NVFP4. Different formats vary in compression method, grouping granularity, and whether scaling factors are preserved, so even if all are called "4-bit," NVFP4 and Q4 differ in actual precision and decode overhead. When one test uses NVFP4 and another uses Q4, the tokens/s they produce are not a like-for-like comparison, and subtracting them directly yields misleading conclusions. Speculative decoding is another layer of variance: a small draft model quickly guesses several tokens, then the main model verifies them in batch, which can significantly speed things up when the acceptance rate is high—but the benefit depends on the task and how well the draft model matches. The article's "MLX with D-Flash 2 speculative decoding 16.6, Ollama Q4 5.7" is exactly the result of changing both quantization and decoding strategy at once, so before trusting any speed comparison, you must confirm whether these two are aligned.

Quick Reference Table and Four Checkpoints

Based on public data under high power plus MLX or Ollama, the rough M4 Max ladder is: Nemotron 3 Nano about 160, Qwen 3.5 35B at 107 to 130, Gemma 4 26B about 75, DeepSeek V4 Flash about 27, dense 27B about 22.

For the fastest coding agent, pick a MoE like Qwen 3.5 or 3.6 35B with MLX or a newer Ollama. For the smartest dense model, pick Qwen 3.8 27B plus speculative decoding. For Gemma 4, runtime barely matters.

Before trusting any tokens/s number, check four things: whether high power mode is on; the runtime and its version; whether the quantization format is NVFP4, 4-bit, or Q4; and whether the result is a single run or the median of several. Miss any one, and the number could be off by three times.

There's also the hidden trap of memory. Jordan points out that Qwen 3.8's 100k-token context consumes about 24GB beyond the model itself, enough to crush a 32GB Mac. A 128GB M4 Max has headroom.

One-line Summary

On the same M4 Max, runtime, power mode, and model architecture can make inference speed differ by three times.

Paragraph Summary

The article aggregates five public benchmarks showing that the same M4 Max running the same model can go from 42 tokens/s to 130 tokens/s, with the gap coming from three commonly overlooked settings. First, runtime: MLX, LLAMA.cpp, and Ollama use different underlying engines, producing 1.5x to 3x differences; Ollama jumped from 58 to 112 after switching to the MLX backend. Second, power mode: low power vs. high power differs by up to 3.84x, a widely ignored variable. Third, model architecture: MoE activates only about 3B parameters per token, far faster than a dense 27B at about 22 tokens/s. The article also offers counterexamples: MLX isn't always twice as fast, leading by only about 10% on most models, and quantization format and memory footprint can skew comparisons.