M5 Max
Apple's second-tier flagship chip in the M5 series, featuring a dual-die design. Two M5 Max chips can be combined into an M5 Ultra through Apple's UltraFusion interconnect technology.
Core Facts
Under the M5 Ultra's 1.2 TB/s bandwidth, the theoretical speed of a 100 GB quantized model is approximately 12 tokens/s, a 200 GB model approximately 6 tokens/s, and a 300 GB model approximately 4 tokens/s.
The M5 Ultra's peak AI compute performance is more than 4x higher than that of the M3 Ultra.
For undergraduate-level AI coursework, an M5 MacBook Pro with 24GB memory and 1TB storage is more than sufficient for tasks such as writing code, running small-scale experiments, and data preprocessing.
Timeline (last 90 days)
Nimble 9B completes a decision in an average of 90 milliseconds on the M5 Max, with only 1 to 4 output tokens and between 155 and 841 input tokens.
Running a 27B model on M1 Max with a 32,000-token prompt requires waiting more than 500 seconds before the first character is output, whereas the M5 Max takes less than one minute (approximately 40 seconds with optimized software).
The decode write speed improvement from M4 Max to M5 Max is only 25%, constrained by memory bandwidth.
The Neural Accelerator requires supporting software to function; running M5 Max with a 2023 old version of llama.cpp yields a read speed of only approximately 990 tokens/s.
The M4 Max of the same name comes in two bandwidth variants—410 GB/s and 546 GB/s—and the M5 Max exists in 460 and 414 versions.
In terms of prompt processing speed, the RTX 4090 achieves approximately 3,000 tokens/s and the M5 Max approximately 800 tokens/s, giving NVIDIA a clear advantage in long-document and codebase input scenarios.
On the M5 Max chip, Lily's prefill speed is 1.23× faster than MLX-LM and its decode speed is 1.35× faster (official data).
The blogger used an M4 Max, an M5 Max, and a Mac Studio to form a three-machine cluster to run the model.
Under the M5 Ultra's 1.2 TB/s bandwidth, the theoretical speed of a 100 GB quantized model is approximately 12 tokens/s, a 200 GB model approximately 6 tokens/s, and a 300 GB model approximately 4 tokens/s.
At a 4K×4K×16K matrix multiplication scale, the M5 Ultra achieved 91 TFLOPS, while the M5 Max and M3 Ultra achieved 34 and 22 TFLOPS respectively.
12 more timeline events
All Facts (20)
Under the M5 Ultra's 1.2 TB/s bandwidth, the theoretical speed of a 100 GB quantized model is approximately 12 tokens/s, a 200 GB model approximately 6 tokens/s, and a 300 GB model approximately 4 tokens/s.
65%VerifiedThe M5 Ultra's peak AI compute performance is more than 4x higher than that of the M3 Ultra.
65%VerifiedFor undergraduate-level AI coursework, an M5 MacBook Pro with 24GB memory and 1TB storage is more than sufficient for tasks such as writing code, running small-scale experiments, and data preprocessing.
65%VerifiedThe M5 Ultra is very likely to adopt UltraFusion dual-chip interconnect packaging technology, combining two M5 Max-level chips into a single logical unit.
65%Partially VerifiedThe UltraFusion inter-chip bandwidth of the M5 Ultra has increased from approximately 2.5 TB/s on the M3 Ultra to 4.4 TB/s, with a 6x improvement in connection density.
70%UnverifiedIn the time-to-first-token test on a 70-billion-parameter model, the M3 Ultra took 31.6 seconds while the M5 Ultra took only 7.7 seconds, achieving approximately 4.1x, close to Apple's claimed 4.3x.
60%UnverifiedNimble 9B completes a decision in an average of 90 milliseconds on the M5 Max, with only 1 to 4 output tokens and between 155 and 841 input tokens.
50%UnverifiedThe decode write speed improvement from M4 Max to M5 Max is only 25%, constrained by memory bandwidth.
50%UnverifiedRunning a 27B model on M1 Max with a 32,000-token prompt requires waiting more than 500 seconds before the first character is output, whereas the M5 Max takes less than one minute (approximately 40 seconds with optimized software).
50%UnverifiedThe M4 Max of the same name comes in two bandwidth variants—410 GB/s and 546 GB/s—and the M5 Max exists in 460 and 414 versions.
50%UnverifiedThe Neural Accelerator requires supporting software to function; running M5 Max with a 2023 old version of llama.cpp yields a read speed of only approximately 990 tokens/s.
50%UnverifiedIn terms of prompt processing speed, the RTX 4090 achieves approximately 3,000 tokens/s and the M5 Max approximately 800 tokens/s, giving NVIDIA a clear advantage in long-document and codebase input scenarios.
50%UnverifiedOn the M5 Max chip, Lily's prefill speed is 1.23× faster than MLX-LM and its decode speed is 1.35× faster (official data).
50%UnverifiedAt a 4K×4K×16K matrix multiplication scale, the M5 Ultra achieved 91 TFLOPS, while the M5 Max and M3 Ultra achieved 34 and 22 TFLOPS respectively.
50%UnverifiedThe M5 Pro's GPU bandwidth and GPU scale are somewhat inferior to those of the M5 Max.
50%UnverifiedFor small-to-medium-scale models, the M5 Max's powerful compute is redundant and does not offer a cost-performance advantage.
50%UnverifiedThe Mac Studio is equipped with the M5 Max chip that debuted in the MacBook Pro in March.
50%UnverifiedThe author considers 12–30 tokens/s to be the comfort zone, with above 30 tokens/s being good; an M5 Max with 48–64 GB can run at approximately 20 tokens/s.
50%UnverifiedQwen 3.8 has a base speed of 22 tokens/s on the M5 Max, which can be boosted to 34 or even 56 tokens/s with MTP enabled.
50%UnverifiedM4 Max的内存带宽可达400GB/s以上
50%