[KongchangAI]
· 2 min read· 1,349 words

M4 Pro vs M5 Pro: Qwen 3.8 Flash Next Prefill Hits 374 Tok/S

M4 Pro vs M5 Pro: Qwen 3.8 Flash Next Prefill Hits 374 Tok/S

A Hacker News user reports M5 Pro hitting 374 Tok/S prefill on Qwen 3.8 Flash Next, with MoEShift seen as the key enabler.

Based on a Hacker News discussion, this article reports that the M5 Pro (24GB) reaches 374 Tok/S prefill when running Qwen 3.8 Flash Next, compared with the M4 Pro. It notes that MoEShift, a dynamic load scheduling strategy for MoE models, may be the key technology behind this speed by optimizing expert migration and caching to reduce scheduling overhead. It also stresses that the original post did not disclose testing conditions, quantization precision, or context length, so reproducibility is uncertain. The discussion's value lies in showing that hardware iteration and inference-scheduling software optimization are advancing in parallel to improve local AI deployment efficiency, reminding developers to focus on framework scheduling strategies rather than relying solely on hardware upgrades.

A Hardware Conversation About MoEShift

When running large language models locally, prefill speed is often the key factor determining how smooth the interaction feels. Recently, a discussion on Hacker News showcased a performance comparison between the M4 Pro and M5 Pro (24GB) chips when running the Qwen 3.8 Flash Next model, and publicly revealed for the first time a prefill speed achieved with the help of MoEShift technology: 374 Tok/S. This figure has drawn considerable attention in the professional community.

For readers unfamiliar with MoEShift, it can be understood as a dynamic load scheduling strategy for Mixture-of-Experts (MoE) models. It allows the model to more efficiently distribute computational tasks across available hardware resources during inference, thereby reducing idle waiting. This discussion centers on the technology's actual performance on Apple chips.

Prefill is one of the two stages of large language model inference, the other being Decode. During the prefill stage, the model processes all of the user's input prompt in parallel, computing the key-value cache (KV Cache) for all tokens at once in preparation for subsequent token-by-token generation. Because this stage can fully leverage the parallel compute of the GPU/Neural Engine, its throughput is typically much higher than that of the decode stage. Prefill speed directly determines how long users wait after submitting a long prompt, which is especially critical for scenarios requiring frequent input of long contexts, such as code completion and document question answering. The unit of measurement is usually Tokens/second (Tok/S), where a higher value means the model "reads through" the input faster.

Core Findings of the Performance Comparison

According to the title information of the Hacker News post, the M5 Pro (24GB) achieved a prefill speed of 374 Tok/S when running Qwen 3.8 Flash Next. The M4 Pro, as the previous generation product, did not have a specific figure given for the same model and configuration, but the post framed the comparison as "vs.", implying a perceptible gap between the two.

The significance of this result lies in the fact that the prefill stage directly determines how fast the model processes input prompts, with an especially pronounced impact on scenarios requiring frequent provision of long contexts (such as code analysis and document question answering). The M5 Pro's ability to reach 374 Tok/S means that, under ideal conditions, the model can more quickly "read through" the user's input and begin generating responses.

You may not have noticed that the post provided only a title and a very brief description, without detailed testing methodology, environment configuration, or raw data points. Therefore, we cannot confirm under what context length, batch size, or quantization precision this speed was measured. This also reminds readers that, in the absence of complete benchmark testing, the absolute significance of a single figure should be treated with caution.

The Role Played by MoEShift

The fact that MoEShift was included in the title indicates that it is not irrelevant decoration but the key enabling technology for achieving this prefill speed. In traditional inference pipelines, different experts of an MoE model need to be activated and allocated to compute units, and scheduling overhead can become a bottleneck. MoEShift reduces repeated loading and communication costs through more fine-grained expert migration or caching strategies.

From an engineering perspective, the combination of this optimization with Apple's unified memory architecture is worth noting. 24GB of memory falls in the "sufficient but not generous" range for running medium-scale MoE models. If MoEShift can dynamically manage expert residency, it may be possible to maintain relatively high throughput under limited memory. However, because the original material does not delve into technical details, we cannot determine whether this technology is an open-source tool, a research prototype, or a patch for a specific framework.

Mixture-of-Experts (MoE) is a sparse architecture in which the feed-forward network layer consists of multiple "expert" sub-networks, and a gating network dynamically selects and activates a small number of experts (usually 2-8) based on input tokens, thereby controlling the computational cost of a single inference while expanding the model's parameter count. However, MoE inference faces the challenges of imbalanced expert load and large memory footprint: different experts need to reside in memory permanently or be frequently swapped in, and if scheduling is improper, expert switching and communication overhead can offset the advantages of sparse computation. On unified memory architectures (such as Apple M-series chips), where the CPU and GPU share a memory pool, expert residency strategies are particularly sensitive to bandwidth contention. Therefore, expert scheduling optimization for MoE (such as dynamic caching, migration, and load balancing) has become an important direction for improving inference throughput.

Implications for Local AI Deployment

The value of this discussion lies not in providing a final conclusion but in revealing a trend: hardware iteration is evolving in sync with inference scheduling technology. The progress of the M5 Pro over the M4 Pro may come partly from the chip's own improved compute and partly from software-layer optimizations such as MoEShift. For local AI developers, this means that simply upgrading hardware is not necessarily the only path; paying attention to the scheduling strategies of inference frameworks is equally important.

In addition, Qwen 3.8 Flash Next as a model-side variable is also worth noting. The Flash series is typically optimized for low-latency scenarios, and its combination with MoEShift may be precisely the source of the 374 Tok/S figure. If more detailed benchmarks are released later, the community will be able to more clearly judge the true magnitude of the M5 Pro's gains.

Qwen 3.8 Flash Next is a model variant in Alibaba Cloud's Qwen series aimed at low-latency scenarios. The "Flash" naming usually implies that this version has undergone inference optimization (such as fewer layers, a more efficient attention mechanism, or quantization-friendly design), trading some accuracy for faster response speed, making it suitable for local deployment and real-time interaction. "Next" may refer to an architectural iteration or a version adapted for a new generation of hardware. When such models are combined with scheduling technologies (such as MoEShift), prefill latency can be further compressed under limited memory, but the specific performance still depends on quantization precision (such as 4-bit/8-bit) and context length, among other undisclosed testing conditions.

Questions Still to Be Verified

At present, the only thing that can be confirmed is this: a Hacker News user reported a prefill speed comparison between the M4 Pro and M5 Pro on Qwen 3.8 Flash Next, in which the M5 Pro reached 374 Tok/S. Beyond that, the testing conditions, reproducibility, and the specific implementation of MoEShift have not been made public.

For readers who plan to use this data as a reference for purchasing decisions or technology selection, it is advisable to remain cautious. Prefill speed is affected by multiple factors, including prompt length, model quantization method, and memory bandwidth usage, and a single figure is not sufficient to represent everyday user experience. Waiting for more complete community reproduction or official benchmarks is the safer approach.

This article is based on publicly available discussions on Hacker News. The data point is a single-source report and does not constitute a performance guarantee.

One-line Summary

A Hacker News user reports that the M5 Pro reaches 374 Tok/S prefill when running Qwen 3.8 Flash Next, with MoEShift viewed as the key enabling technology.

Paragraph Summary

Based on a Hacker News discussion, this article reports that the M5 Pro (24GB) achieves a prefill speed of 374 Tok/S when running the Qwen 3.8 Flash Next model, compared with the M4 Pro. The article notes that MoEShift, as a dynamic load scheduling strategy for Mixture-of-Experts models, may be the key technology enabling this speed by optimizing expert migration and caching to reduce scheduling overhead. It also emphasizes that the original post did not disclose testing conditions, quantization precision, or context length, leaving the reproducibility of the data in question. The value of the discussion lies in revealing that hardware iteration and inference-scheduling software optimization are advancing in parallel to improve local AI deployment efficiency, reminding developers to pay attention to framework scheduling strategies rather than relying solely on hardware upgrades.

#M4 Pro#M5 Pro#Qwen 3.8 Flash Next#MoEShift#prefill speed#local AI deployment#Mixture-of-Experts
Share:

Related articles