Ollama v0.40.0 Released: MLX Runtime Now Default on Apple Silicon

Ollama, one of the most popular tools for running large language models locally, has surpassed 182,000 stars on GitHub. Its latest v0.40.0 pre-release brings an important improvement for Apple device users: on Apple Silicon chips, the MLX runtime is now enabled by default for model inference.

MLX Becomes the Default Runtime on Apple Silicon

The core change in this update is straightforward: on Apple Silicon devices, any model architecture supported by the MLX runtime will automatically run on MLX. For Mac users, this means getting inference performance deeply optimized for Apple chips — no extra configuration required.

MLX is Apple's machine learning framework, designed specifically for Apple Silicon's Unified Memory Architecture. Compared to general-purpose inference backends, it can more effectively leverage the CPU, GPU, and Neural Engine resources of M-series chips, typically delivering better memory efficiency and inference throughput. By making MLX the default, Ollama's team is signaling that it views the Mac platform as a first-class environment for local inference.

The official usage example is refreshingly simple — users can pull and run models with familiar commands:

ollama pull qwen3.8
ollama run qwen3.8

As the example shows, popular models like the Qwen3 series are already within the supported range.

Apple Silicon's Unified Memory Architecture (UMA) is the key backdrop for understanding MLX's advantages. In a traditional PC, the CPU and GPU each have their own separate memory pools, and transferring data between them creates bandwidth bottlenecks and latency. M-series chips integrate the CPU, GPU, Neural Engine, and memory into a single package, allowing all compute units to share the same high-speed memory pool — data can be accessed directly by different compute units without copying across a bus. This is hugely significant for LLM inference: model weights only need to be loaded once and can be seamlessly shared between the CPU and GPU, with memory bandwidth fully utilized rather than wasted on copy overhead. The MLX framework is designed from the ground up around this architectural characteristic, intelligently scheduling tensor operations across different compute units. General-purpose inference backends — particularly those ported with a CUDA mindset — typically cannot fully exploit this architectural advantage.

A Gradual Rollout During Pre-release

It's worth noting that v0.40.0 is currently marked as a Pre-release. Ollama has explicitly stated that it will continue testing during this phase and progressively enable MLX runtime support for more model architectures.

This incremental strategy is a prudent one. Different model architectures vary in how well they adapt to a given runtime, and switching inference backends by default — if handled carelessly — can introduce compatibility or precision issues for certain models. By starting with verified architectures during the pre-release phase and gradually expanding the support list, the team can strike a balance between performance gains and stability.

For everyday users, this also means that if a particular model isn't yet running on the MLX path, that's an expected part of the transition period before the stable release. Keep an eye on upcoming release notes for updates to the supported model list.

The compatibility challenge for inference backends largely stems from the diversity of model architectures. Different models vary in their attention mechanisms, activation functions, and normalization approaches — for example, GQA (Grouped Query Attention), RoPE (Rotary Position Embedding), SwiGLU activations, and so on. Each variant requires the runtime framework to provide corresponding operator implementations. MLX's operator library is still being actively expanded, and for some newer or less common architectures, fully validated implementations may not yet exist. Forcing MLX onto an unsupported architecture could result in performance degradation (falling back to an inefficient code path) or even numerical precision errors that cause anomalous model output. For this reason, Ollama uses an architecture allowlist approach — automatically enabling MLX only for verified model architectures, while continuing to use the existing llama.cpp backend for everything else. This is a common engineering strategy for preserving a stable user experience when introducing new features.

What This Means for the Local Inference Ecosystem

Running large language models locally is becoming an increasingly common choice for developers and individual users alike — both for data privacy reasons and to avoid paying for cloud API usage. Ollama has carved out an important niche in this space thanks to its clean command-line experience and broad model support.

By making MLX the default runtime on Apple Silicon, this update further lowers the bar for Mac users to achieve high-performance local inference. In the past, fully leveraging MLX on a Mac often required dedicated toolchains or manual configuration. Now, that capability is baked directly into Ollama's default behavior.

Community response to the release has been enthusiastic — the release page has drawn a large number of thumbs-up, celebration, and rocket emoji reactions, with 28 people participating. This speaks to the long-standing demand among Mac users for native performance optimization.

Summary

Ollama v0.40.0 is a platform-focused update, with its headline feature being MLX-accelerated inference as the default experience for Apple Silicon users. As a pre-release, it's still expanding support to cover more model architectures. For users doing local AI experimentation and development on a Mac, this is definitely a worthwhile upgrade to explore — just keep an eye on the official changelog to track which models are supported as the rollout continues.