Which Mac Runs Which LLM Fastest? Real Benchmark Data Tells You

Why Is There Such a Huge Speed Gap Between Macs Running LLMs?

Recently, a project that sparked heated discussion appeared on Hacker News: Which Mac runs which LLM, and how fast. The author did something many people wanted to do but didn't have time for—putting different Mac models under the same benchmarking setup to test their actual speeds running various local large language models.

Apple Silicon's unified memory architecture has made Macs a popular choice for running local LLMs, but there's a massive gap between "can run" and "runs fast." The same 7B model might perform several times differently on a base M1 versus an M3 Max. This project attempts to answer a very specific question with data: What size model is your Mac actually suited for?

Testing Methodology and Core Findings

Side-by-Side Comparison Under a Unified Benchmark

The value of this project lies in controlling variables. Different Macs have varying memory bandwidth, GPU core counts, and neural engine specs—all of which directly affect token generation speed. The author fixed model size, quantization precision, and inference framework, changing only hardware variables to produce comparable conclusions.

Based on community discussion, the tests covered multiple tiers from base M series to high-end Max chips, with models ranging from common 7B and 13B to larger open-source models. This matrix-style testing is more valuable for reference than scattered individual benchmarks.

Memory Bandwidth Matters More Than Compute Power

LLM inference is a classic memory-bandwidth-bound task. Generating each token requires moving model weights from memory to compute units, so higher bandwidth means faster speed. This explains why the M3 Max often far outperforms the M3—the GPU core count gap between them isn't as dramatic as the bandwidth gap.

Developers in the Hacker News comments pointed out that for local inference, memory capacity determines "whether it can run," while memory bandwidth determines "how fast it runs." This judgment aligns with the direction of the test results and is the most important spec to watch when buying a Mac for local AI development.

Memory bandwidth refers to the maximum rate at which the processor reads data from memory, typically measured in GB/s. During LLM autoregressive generation, every token produced requires a full traversal of model weights, and the computation relative to data movement is minimal—so throughput is almost proportional to bandwidth rather than peak compute. Base M3 bandwidth is around 100 GB/s, while M3 Max can reach 300–400 GB/s, and this gap maps directly to tokens/s. This is also why CPU inference isn't inferior to GPU on some small models—the bottleneck is always memory, not compute units. When evaluating whether a Mac is suitable for local inference, prioritize the bandwidth tier corresponding to the chip model rather than just counting GPU cores.

Unified Memory Architecture (UMA) is the core design of Apple Silicon: CPU, GPU, and neural engine share the same physical memory, with no need to copy data between VRAM and system memory via PCIe. For LLM inference, this means model weights only need to be loaded once and can be directly accessed by the GPU, avoiding the common VRAM shortage issues of discrete graphics cards. The trade-off is that total memory is fixed and non-upgradeable, making memory capacity a one-time decision at purchase. This also explains why a Mac's "available VRAM" roughly equals total system memory, while a PC with a discrete GPU at the same price often has only 8–16 GB VRAM.

Practical Advice for Developers and Regular Users

If you're considering running local models on a Mac, you can decide based on three dimensions.

Budget first: Base M series are suitable for 7B-level quantized models—adequate for daily Q&A and simple code completion, but don't expect smooth performance on larger models.

Efficiency first: The Max series' high bandwidth advantage is evident in long-context and larger-model scenarios, making it suitable for developers who frequently call local models.

Capacity first: If your goal is running 30B+ models, memory capacity is a hard threshold—no amount of bandwidth can compensate for not being able to fit the model.

Interestingly, quantization technology is rapidly lowering hardware barriers. 4-bit quantization halves the memory footprint of many models, making the practical usable range of mid-range Macs much wider than a year ago.

Quantization is the technique of compressing model weights from 16-bit floating point to 8-bit, 4-bit, or even lower precision. Taking 4-bit as an example, a 7B model's memory footprint can drop from about 14 GB to around 4 GB, allowing even base Macs to load it. Common formats include GGUF (used with llama.cpp, Ollama) and GPTQ/AWQ (used with GPU inference frameworks). Quantization introduces some precision loss—perplexity increase is typically limited at 4-bit, but overly low bit widths can noticeably affect reasoning and code task performance. When choosing a quantization level, balance "fits in memory" against "answers accurately" rather than blindly pursuing the smallest size.

Where Are the Limits of Local Inference?

Discussion of this project also exposed the real limitations of local LLMs. Even top-spec Macs still can't match cloud GPU clusters on very large models—the advantages of local inference lie in privacy, offline availability, and zero marginal cost, not absolute performance.

Some in the community also believe that with advances in model distillation and miniaturization, the local experience on mid-range hardware will continue to improve. For individual developers and small-to-medium businesses that prioritize data privacy, local Mac deployment remains a route worth seriously evaluating.

This Show HN project itself isn't complex, but it turns scattered experience into a queryable data table—exactly the kind of contribution the developer community needs most.

Token generation speed is typically divided into two metrics: the prefill stage processes the entire prompt and is compute-intensive, heavily affected by compute power; the decode stage generates tokens one by one and is memory-bandwidth-intensive. What users perceive as "typing speed" is mainly the decode rate, generally expressed in tokens/s. As a rule of thumb, above 20 tokens/s approaches reading speed for smooth interaction; below 5 tokens/s is noticeably laggy. Long-context scenarios amplify prefill overhead, which is also why high-bandwidth and large-memory machines show more obvious advantages. When evaluating benchmark data, distinguish between these two stages and avoid summarizing the overall experience with a single number.

One-line Summary

A unified benchmark measuring LLM speed across Mac models reveals that memory bandwidth is the key bottleneck.

Paragraph Summary

The open-source Hacker News project "Which Mac runs which LLM, and how fast" benchmarked local large language model speed across multiple Mac tiers—from base M series to high-end Max chips—under fixed model size, quantization precision, and inference framework. The article points out that LLM inference is a classic memory-bandwidth-bound task: memory capacity determines "whether it can run," while bandwidth determines "how fast it runs," explaining why M3 Max significantly leads M3. Based on this, the article offers three purchasing approaches—budget first, efficiency first, and capacity first—and discusses how quantization lowers hardware barriers, as well as the advantages of local inference in privacy and offline availability versus its limitations in absolute performance.