Ollama v0.40.6 Released: Adds q2_0 Quantization and Improves Windows Stability

Ollama has released the v0.40.6 pre-release version. Although this update contains only a few changes, it touches on three areas: quantization format expansion, Windows platform stability, and cloud Code assistance features, all of which have practical implications for developers and operations staff running local models with Ollama.

Windows Fix for Orphaned Processes

In this update, a change submitted by contributor dhiltgen (#18500) focuses on the Windows platform: preventing runner processes from becoming orphaned after an abnormal exit and continuing to consume resources.

When running Ollama on Windows, if the main process is forcibly closed or crashes, the runner subprocess responsible for inference sometimes does not exit along with it, but instead remains in the system continuing to occupy VRAM and memory. For users who frequently start and stop models or run long-term stress tests, such residual processes gradually accumulate, eventually preventing new models from loading properly. This fix makes process lifecycle management under Windows cleaner and represents an important stability patch.


The difference between orphaned processes and zombie processes: An orphaned process refers to a child process that is still running after its parent has exited; in that case, the child is usually re-parented under another process by the system. A zombie process refers to a child that has already terminated but whose parent has not reaped its exit status, so its process table entry remains occupied. The runner residue described here is closer to the former type of resource leak: after the main process (ollama serve or CLI) is forcibly closed, the runner subprocess responsible for actual inference is not notified to exit and continues holding VRAM and memory. Windows and Unix have different process management models, and Windows lacks POSIX signal propagation semantics, so cross-platform code often needs to explicitly chain the parent and child process lifecycles on Windows. This also explains why such issues often surface on Windows first.

Added q2_0 Tensor and File Type

Another core change comes from jmorganca (#18967): adding q2_0 tensor type and corresponding file type support for the GGUF format.

q2_0 is an extremely low-bit quantization scheme, representing each weight with only about 2 bits. The significance of this format is that it further compresses model size, giving larger-parameter models a chance to be loaded on devices with limited VRAM or memory. The tradeoff is that accuracy loss is noticeably higher than with common quantization levels such as q4 and q5, so it is better suited to scenarios with higher tolerance for answer quality, such as simple classification, local experiments, or prototype validation on edge devices.

From an engineering perspective, supporting q2_0 means Ollama's quantization coverage continues to extend toward the low-bit end. Combined with the existing q4/q5/q8 options, users gain finer-grained room to trade off between model size and quality.

What quantization and bits are: Model weights are stored by default as 16-bit floating point (FP16). Quantization approximates these weights using fewer bits, thereby compressing size and reducing memory bandwidth requirements with almost no change to computation results. A common approach groups weights into blocks that share a scaling factor, with each weight in the block retaining only a few bits. The lower the bit count, the smaller and faster to load the model is, but the greater the representation error per weight becomes; accumulated across the entire network, this shows up as worse answers, broken logic, or repetition. q4, q5, and q8 represent roughly 4, 5, and 8 bits per group of weights, respectively, while q2_0 compresses it to about 2 bits, roughly twice the compression rate of q4.

GGUF and tensor types: GGUF is the model file format led by the llama.cpp ecosystem, and Ollama directly reuses its tensor type enum to define quantization methods. Each quantization scheme corresponds to a TensorType number in the file (such as Q4_K, Q5_K), and the loader chooses the corresponding dequantization kernel based on it. Adding a new q2_0 tensor type means Ollama's file parsing layer and inference kernel both need to recognize and correctly decode this new encoding; otherwise, the model will error out directly during loading.

Cloud Code Assistance and --ask Mode Improvements

jberg5 brought two changes related to developer experience. One is "Launch clef in Ollama's cloud" (#19036), which brings clef-related capabilities into Ollama's cloud; the other is "use identifiers for --ask mode question keys" (#19041), which changes question keys in --ask mode to use identifier handling.

These two changes lean toward internal mechanisms and cloud service integration. Ordinary users may not notice them directly, but if you are using Ollama's cloud capabilities or the --ask interactive mode, adjustments to key handling will affect the stability and consistency of request construction.

Version Information and Upgrade Advice

This version was released by github-actions on October 16, with commit 2476d5e and a GitHub verified signature (GPG key ID: B5690EEEBB952194). The full change comparison can be viewed from v0.40.5 to v0.40.6-rc0. Contributors to this release include jmorganca, dhiltgen, and jberg5.

It is worth noting that it is currently marked as Pre-release, meaning it has not yet entered the official stable channel. If your production environment depends on Ollama, it is advisable to first verify q2_0 model loading behavior and Windows process management behavior on a test machine before deciding whether to follow up. For users who just want to try low-bit quantization, you can pull this pre-release version separately for experiments without replacing your existing stable version.

Overall, v0.40.6 is a release oriented toward maintenance and format expansion. It does not introduce breaking changes, and its main value lies in the Windows stability fix and the expansion of quantization format boundaries.

One-line Summary

Ollama v0.40.6 pre-release: fixes residual Windows processes, adds q2_0 ultra-low-bit quantization, and improves cloud and --ask mode.

Paragraph Summary

Ollama has released the v0.40.6 pre-release version, with changes concentrated in three areas. First, Windows stability: it fixes the issue where runner subprocesses remain after the main process crashes or is forcibly closed, continuing to occupy VRAM and memory (#18500), which is especially important for users who frequently start and stop models or run long-term stress tests. Second, quantization format expansion: GGUF gains q2_0 tensor and file type support (#18967), at about 2 bits per weight, further compressing model size at the cost of noticeably higher accuracy loss than q4/q5, making it suitable for scenarios with higher tolerance. Third, developer experience: it brings clef capabilities into Ollama's cloud and switches --ask mode question keys to identifier handling. This version is a Pre-release, and the author recommends verifying it in a test environment before deciding whether to follow up.