Ollama v0.40.10-rc0: Gemma 4 Renderer Update and MLX Speculative Decoding Enhancements

A Low-Key Release Hiding a Key Piece of the Gemma 4 Ecosystem

Ollama has released the v0.40.10-rc0 pre-release (commit f52b9e7), with core changes centered on the renderer and inference support for the Gemma 4 model family. Judging from the commit messages, this update is not a simple bug fix but rather fills in two pieces of infrastructure for Gemma 4 in local inference scenarios: a dedicated renderer for agent tool calling, and speculative decoding support on the MLX backend.

For developers focused on local LLM deployment, commit-level information like this often reveals the product's direction more clearly than official blog posts.

gemma4-nemotron: Adjusting the Render Order of Assistant Content and Tool Calls

Why Render Order Matters

In multi-turn tool-calling scenarios, the order in which the model outputs assistant text and initiates tool calls directly affects how the chat template is parsed. Some models tend to output thinking or explanatory text first and then initiate tool calls; others do the opposite.

According to the commit description, the newly added gemma4-nemotron renderer renders assistant content before tool calls, rather than after the tool response. This change targets Gemma 4 variants that use the Nemotron-style chat format.

Chat templates and renderers are the step where local inference tools assemble structured messages (system/user/assistant/tool roles) into a prompt the model can read. The same multi-turn conversation may require different separators, role prefixes, and field orders depending on the model's training format. The renderer is the implementation of this conversion logic, typically driven by Jinja templates.

Render order is an "implicit contract" because it is not reflected in the model weights, yet it directly determines whether the token sequence the model sees matches what it saw during training. Once the order of assistant text and tool-call blocks is misaligned, the model may misinterpret tool results as its own output, leading to tool-call loops or parsing failures. These issues often only reproduce in specific multi-turn scenarios, showing up in logs as inexplicable format errors that are costly to debug. That is why pairing renderers with chat template fixtures and running Jinja render parity checks is a practical way to prevent regressions.

Explicit Declaration via RENDERER

Models can explicitly select this rendering backend via RENDERER gemma4-nemotron. Crucially, the auto-detection logic for gemma4 remains unchanged—meaning the default behavior of existing models will not be disrupted, and the new renderer is a purely incremental addition.

The commit also adds a corresponding chat template as a test fixture for this renderer, used for Jinja render parity checks. This is commendable: once an "implicit contract" like a renderer goes wrong, debugging is extremely costly, and having a parity check as a safety net significantly reduces regression risk.

Explicit opt-in is a common compatibility strategy in model integration: new behavior is not made the default but requires model metadata to actively enable it via a declaration like RENDERER <name>. The benefit is that the rendering path of existing models is completely unaffected, preventing a version upgrade from silently changing the output format of existing deployments.

The alternative is auto-detect: the tool infers which renderer to use based on model architecture or template features. Auto-detection is convenient but risky—once the inference rules change, it may inadvertently affect a batch of models that should not be affected. By keeping gemma4's auto-detection logic unchanged and making the new renderer a purely incremental entry point, Ollama is essentially balancing "default stability" with "new capabilities available"—a typical approach for open-source inference tools dealing with model family fragmentation.

MLX Backend: Enabling DFlash Speculative Decoding for Gemma 4

Loading the Draft Output Head on Demand

The second change focuses on the MLX inference path. The commit description mentions that when the draft model's configuration requires it, the draft's own output head will be loaded, while retaining the target-head path used by existing DFlash models. This amounts to compatibility handling between two output-head loading strategies, preventing new model integration from breaking the operation of older models.

Exposing Hidden States and Raw Logits

To support speculative decoding, this version exposes hidden-state taps and raw logits for Gemma 4. Speculative decoding relies on alignment between the draft and target models at the intermediate layers and output distributions, so the visibility of these internal tensors is a prerequisite for achieving acceleration.

Another detail is that safetensors target precision is now reported independently of the quantized draft weights. This change seems trivial but is quite useful for debugging precision issues under quantization combinations—previously, precision information could easily be confused by the draft's quantization state.

Speculative Decoding is an inference technique that accelerates autoregressive generation without changing the output distribution. The basic process is: a smaller, faster draft model first "guesses" the next several tokens, then the target model verifies these candidate tokens in parallel in one pass; the accepted prefix is kept directly, and after the first rejected position, it falls back to standard token-by-token sampling. Since the verification phase can be batched, overall throughput typically improves by 1.5–3x, and theoretically the results are equivalent to the original model's greedy/sampling outputs.

For this mechanism to work, the draft and target must be aligned in vocabulary and output space, so draft models often need to reuse the target model's output head (target-head) or carry their own matching output head. The "on-demand loading of the draft output head" and "retaining the target-head path" mentioned in the article are precisely for compatibility with these two types of draft structures. The exposure of hidden-state taps and raw logits allows the verification phase to access intermediate representations for finer alignment and rejection decisions.

What the Commits Reveal About Ollama's Model Integration Strategy

Looking at the two changes together reveals a methodology in Ollama's model integration: at the renderer level, explicit opt-in plus automated testing ensures compatibility; at the inference level, a dual-path loading strategy accommodates both new and old models.

Gemma 4 ecosystem adaptation is still in the groundwork phase. Changes like render order, speculative decoding, and precision reporting are not directly user-facing, yet they determine whether future models can work "out of the box." The rc designation also indicates these changes are still being validated, and the official release may see further adjustments.

For teams building their own inference services, it is worth paying attention to the chat template changes in this version, especially for Gemma 4 deployments that use tool calling.