Ollama v0.40.7-rc0 Adds PPLX Decider v1.2 Support

A Seemingly Routine MLX Update Hides Ambitions for an Inference Scheduling Layer

Ollama pushed the v0.40.7-rc0 pre-release tag on GitHub, commit 8be580a, tagged by maintainer dhiltgen. Judging from the title, this is an incremental update to the MLX (Apple Silicon machine learning framework) backend: adding support for PPLX Decider v1.2.

At first glance, this commit message looks like an ordinary dependency upgrade and is easy to skip past. But the implementation details revealed in the commit body show that things are not so simple.

What Exactly Is PPLX Decider

The commit description states:

Implement the Qwen3.5-backed decision scorer with noncausal attention, the publisher readout, image handling, and System One prompt encoding.

Breaking this down, there are several key pieces of information:

The decision scorer is powered by Qwen3.5. PPLX Decider is not a standalone small model, but a decision scorer based on Qwen3.5. The term "decider," in the context of inference systems, typically refers to a scheduling component that determines "which path to take next"—such as whether to trigger a tool call, which reasoning branch to select, or how to allocate compute budget.

Noncausal attention. This is a strong signal. Conventional autoregressive generation uses causal attention, where each token can only see the context preceding it; noncausal attention allows tokens to be visible in both directions. This typically appears in model heads used for discrimination, scoring, or classification, rather than pure generation tasks. In other words, this Decider is more of an "evaluator" than a "generator."

Publisher readout. This can be understood as the readout head for the model output, responsible for mapping hidden states to decision scores.

Image handling and System One prompt encoding. This indicates that the decider can accept image inputs and employs a prompt encoding scheme called "System One." The name System One evokes the fast, intuitive decision path in dual-system theory—quite fitting for low-latency routing decisions.

Noncausal attention and bidirectional encoding: Noncausal attention (noncausal / bidirectional attention) removes the causal mask, allowing every position in the sequence to attend to the full context. BERT-style encoders adopt this structure and excel at producing global semantic representations, making them suitable for classification, ranking, and scoring tasks. Using it for the Decider means the module performs a single bidirectional encoding of the complete input (including image tokens and prompt text), then outputs decision scores through a readout head—a typical discriminative use rather than token-by-token autoregressive generation. This aligns closely with the scheduling need to "decide which path to take"—a decision requires only a single forward pass, no decoding loop, and lower latency.

System One prompt encoding: In dual-system theory, System 1 refers to fast, automatic, low-cognitive-load judgments, while System 2 is slow, deliberate reasoning. Naming the prompt encoding scheme System One suggests that this encoding is oriented toward short-path, low-latency routing decisions, possibly using compressed prompt templates or a lightweight tokenizer to complete encoding in milliseconds on-device.

Why MLX Was Chosen as the First Deployment Backend

This update sits under the mlx namespace, meaning the capability lands first on the Apple Silicon platform. MLX is a framework designed by Apple for unified memory architecture, with clear advantages for running small and medium-sized models on M-series chips.

Prioritizing on-device deployment for the decision scorer makes logical sense: routing decisions require extremely low latency, and local deployment is more reasonable than round-tripping to the cloud. Using Qwen3.5 as the base model indicates a preference for lightweight, locally deployable models.

Combined with Ollama's recent positioning evolution—expanding from "a simple tool for running models locally" to "a local inference runtime"—introducing a scheduling decision layer is a natural step. Getting models to run is only the foundation; deciding when to invoke which model and which reasoning path to take is the core competitiveness at the runtime level.

MLX's unified memory and latency advantages: MLX adopts a unified memory model, where the CPU and GPU share the same physical memory, eliminating the need for explicit tensor copies between host and device as with CUDA. For a decision scorer that frequently performs forward computations on small-batch inputs, this zero-copy characteristic can significantly reduce end-to-end latency per inference. In addition, MLX uses lazy evaluation graphs, with operator fusion completed at compile time, making it suitable for workloads like routers that have fixed computation graphs and very small batch sizes.

What PPLX means: PPLX is typically an abbreviation related to Perplexity, and in the context of inference frameworks it often refers to components related to perplexity scoring or routing selection. Naming it Decider suggests that the module selects among multiple candidate paths through some scoring mechanism, rather than hard-coded rules.

What a Pre-release Version Means

The rc0 suffix indicates that this is the first iteration of a release candidate. dhiltgen tagged it on October 17, and the commit itself carries a GitHub-verified signature (GPG key ID: B5690EEEBB952194), confirming a trustworthy source.

For ordinary users, this version will not bring visible changes in the short term, and PPLX Decider is likely still in internal enablement or experimental stages. But for developers following local inference architecture, this commit is worth noting: it suggests that Ollama is building an intelligent scheduling capability beyond models, not just a model loader.

Points worth watching going forward include: whether PPLX Decider will follow on non-MLX backends (CUDA, ROCm); whether System One prompt encoding will become an independent specification; and whether the "decider" abstraction will expand into a pluggable routing policy system. Currently, public information comes only from a single commit, with no documentation or design notes to corroborate it, so the above judgments should be treated with caution.