Ollama v0.40.7-rc1: MLX and GGML Share CUDA Runtime, Slimmer Package Size

One "dedup," size optimization for two backends

Ollama released v0.40.7-rc1, and the core change is summed up in a single headline: mlx: deduplicate CUDA runtime payloads. This change looks unremarkable, but it points to a long-standing engineering pain point for local inference tools—when a single distribution package contains both the GGML and MLX backends, the CUDA runtime libraries get packaged twice, which wastes space and complicates dependency resolution.

The commit was made by dhiltgen and carries a GitHub verified signature. The change itself revolves around three actions: move, retain, and verify.

Breaking down the change: which libraries move, which stay

Shared libraries migrate to a "sibling directory"

The commit message is explicit: the CUDA libraries shared by the GGML and MLX backends are moved into a sibling CUDA payload directory. In other words, the CUDA runtime that each backend previously carried separately is now stored in one place.

The loading logic is adjusted accordingly: whether loading MLX in-process or in a subprocess, it must be able to resolve to this bundled dependency directory. This is a classic "change the structure, change the loader" chain of modifications, and it explains why a dedup commit touches both the packaging and runtime paths at once.

CUDA runtime payload refers to the set of CUDA runtime components distributed with the release package, typically including dynamic libraries such as libcudart, libcublas, libcublasLt, and libcudnn, along with their dependencies. They are substantial in size—a single library often reaches hundreds of MB—so carrying two copies has a significant impact on package size. Ollama's packaging approach organizes these components into a separate payload directory, which the loader locates on demand at startup and adds to the dynamic linking search path (such as LD_LIBRARY_PATH or the platform equivalent). "Sibling directory" means the payload sits under the common parent of the respective GGML and MLX backend directories, so both backends can reference the same file via a relative path, physically keeping only one copy.

MLX private libraries and JIT headers stay in place

Deduplication does not mean merging everything. The commit specifically notes that MLX-private libraries and JIT headers remain in the MLX overlay. These files are unique to the MLX backend; sharing them with GGML makes no sense, and forcing a merge would introduce coupling instead.

The boundary is drawn clearly: shared files go into the common directory, private files stay in the overlay. Deciding what stays or goes based on ownership rather than size is a relatively sound approach in packaging engineering.

MLX overlay is a layering concept in Ollama's packaging system: the base layer provides the common runtime, and the overlay layer adds files specific to a given backend. MLX's JIT (just-in-time compilation) headers cannot be shared because, on some platforms, MLX uses these headers at runtime to compile and generate kernel code; the header version must strictly match the MLX library itself. If they were overwritten by the GGML side or another version of MLX, it could cause compilation failures or inconsistent behavior. Likewise, MLX-private dynamic libraries (such as libmlx) are linked only by the MLX backend; putting them in the common directory brings no benefit and would make the GGML path unexpectedly depend on them. Keeping such files in the overlay effectively hard-codes the "who owns it, who uses it" boundary in the directory structure.

OpenBLAS Fortran runtime closure fully included

An easily overlooked detail: the commit requires including the complete OpenBLAS Fortran runtime closure. A closure means not just the top-level library but also the runtime components it depends on must be packaged together. MLX depends on OpenBLAS on some non-macOS platforms; if the closure is incomplete, missing symbols may only surface at runtime—and such issues are often discovered by users after release.

Runtime closure is a packaging term referring to the full set of dynamic libraries consisting of a library and its recursive dependencies. Take OpenBLAS: it depends on Fortran/GCC runtime libraries such as libgfortran, libquadmath, and libgcc_s. If only libopenblas is packaged and its dependencies are omitted, the program works on a developer machine because the system already has these libraries installed, but in a clean user environment it reports undefined symbol or cannot open shared object file at load time or on first call. Requiring a "complete closure" means the packaging script must recursively collect along the dependency graph rather than merely copying directly referenced files. Such issues are subtle and delayed, so explicitly verifying closure completeness in the release process is a necessary safeguard.

Dependency verification added to the release packaging stage

The change also requires verifying the above dependencies in the release packaging process. After deduplication, the dependency relationship shifts from "each backend carries its own" to "provided by a common directory," and any packaging omission could cause a backend to fail to load on a user's machine. Putting the verification into the release packaging step provides a pipeline-level safety net.

GGML and MLX are Ollama's two inference backends. GGML is the underlying tensor computation library of the ggml/llama.cpp ecosystem, implemented in C/C++, cross-platform, and supporting CPU and various GPU accelerations (CUDA, Metal, Vulkan, etc.); it is Ollama's default path for running models on most platforms. MLX is an array computation framework designed by Apple for Apple Silicon, fully leveraging the unified memory architecture and typically delivering better performance on M-series chips. Ollama introduced the MLX backend on macOS as an optional or default path, so a single distribution package must accommodate both backends. Although their computational graph implementations differ, both may call the same set of underlying runtimes (such as CUDA-related libraries) on certain platforms—this is exactly the root cause of duplicate packaging.

Practical impact on users

For ordinary users, the most direct result of this change is that duplicate CUDA runtime copies no longer exist in the distribution package, reducing download size and disk usage accordingly. For Apple Silicon users of the MLX backend, the adjusted loading path requires the loader to correctly resolve the new dependency directory—which is why the commit modifies both in-process and subprocess loading logic.

For release maintainers, an extra layer of packaging verification means a longer pre-release checklist, but in exchange for more predictable runtime behavior.

Engineering perspective: why such changes deserve attention

The competitiveness of local inference tools largely depends on whether the distribution package is "clean." GGML and MLX evolve separately, yet they share more and more underlying dependencies, and duplicate packaging will keep growing with each version iteration. This commit chooses to deduplicate at the CUDA payload level rather than simply deleting files, showing that the maintainers are controlling the scope of coupling—shared stays shared, private stays private, and release verification locks in the result.

This is typical release engineering work; it won't appear prominently in feature changelogs, yet it directly affects installation success rates and package size. As a release candidate, v0.40.7-rc1 is still in the verification stage, and final behavior is subject to the official release.

Key takeaways

• CUDA shared libraries move to a sibling CUDA payload directory; GGML and MLX no longer each carry a copy
• MLX private libraries and JIT headers remain in the MLX overlay to avoid unnecessary coupling
• Complete OpenBLAS Fortran runtime closure is packaged, reducing the risk of missing symbols at runtime
• In-process and subprocess MLX loading logic is adjusted in sync to resolve the new dependency directory
• Dependency verification is added to the release packaging stage as a pipeline safety net

One-line Summary

Ollama shrinks its distribution package and reduces dependency risk by deduplicating the CUDA runtime and drawing the line between shared and private.

Paragraph Summary

The core change in Ollama v0.40.7-rc1 is deduplicating the CUDA runtime libraries carried redundantly by both the GGML and MLX backends, consolidating them into a sibling CUDA payload directory, and adjusting the in-process and subprocess MLX loading logic in sync to resolve the new path. The change is not a blanket merge: MLX private libraries and JIT headers remain in the MLX overlay, with decisions based on ownership rather than size; it also requires packaging the complete OpenBLAS Fortran runtime closure and adds dependency verification in the release packaging stage as a safety net. For users, download size and disk usage decrease; for maintainers, the pre-release checklist grows longer but runtime behavior becomes more predictable. This is a typical release engineering optimization, embodying the principle of "shared stays shared, private stays private, and verification locks in the result."