Ollama's Decision Model: Running Nimble Locally with --ask

Ollama has tucked an interesting experimental feature into the terminal with version 0.40.4: the decision model Nimble. Its pitch is straightforward—instead of asking the model to write a paragraph, you have it directly answer yes/no, return a label, or produce a score with a probability. The command is just as simple:

ollama run nimble --ask

The signal here is clear: how large models are used in the terminal is branching off from "conversation" into a separate "decision" line.

Why a "Decision Model" Instead of a Chat Model

Most everyday use of Ollama for running local models is Q&A and completion. But there's a whole class of needs in real workflows that has long been overlooked: you just need a judgment.

For example, whether content moderation should block something, which label a customer support ticket should get, what the risk score of a piece of text is, whether a particular rule is triggered. Chat models can do these tasks too, but the cost is that the model first outputs an explanation, then gives a conclusion. For scripted calls, parsing that natural language is itself a burden.

The design philosophy behind "Jev-style" decision models like Nimble is to compress the output space down to something tiny: yes/no, a finite label set, a probability between 0 and 1. This brings two benefits. First, the output is parseable and goes straight into a program; second, inference overhead is more controllable because you don't need to generate large blocks of text.

Ollama exposes this kind of interaction explicitly through the --ask flag, effectively admitting at the CLI level that decision-making is a distinct invocation mode, not a degenerate version of chatting.

Jev-style Decision Models: Here "Jev" refers to a design paradigm centered on an extremely minimal output space (similar to the Jevons paradox idea of "achieving a clear goal with limited means"). The key difference from conventional instruct/chat models lies in the training objective and output constraints. Ordinary chat models are optimized during the RLHF stage to be "helpful, harmless, honest," so they tend to generate explanatory text. Jev-style decision models, by contrast, use large numbers of labeled samples (such as yes/no, category IDs, probability values) during fine-tuning to directly push the output distribution toward discrete enumerations or scalars, often combined with logit constraints or finite-token vocabularies, making it nearly impossible for the model to output extraneous text.

These models usually still use a base LLM as their backbone, so they retain semantic understanding, but at inference time they only need to decode 1–2 tokens, with latency and VRAM usage far lower than free-form generation. The trade-off is that they can't handle open-ended Q&A and aren't good at tasks requiring multi-step reasoning. Understanding this makes it clear why Nimble is exposed separately with --ask rather than reusing the default chat mode of ollama run.

--ask's Actual Behavior and Verification Commands

According to the original information, --ask mode supports three question forms:

• Yes/no questions: the model returns a binary judgment
• Label questions: select a label from a given set of categories
• Scoring questions: output a probability value

A typical invocation looks like this:

ollama run nimble --ask "Does this comment contain advertising?"

Note that this material itself comes from RSS with low information density; it doesn't provide a complete parameter list, temperature setting recommendations, or batch invocation methods. So what's shown above is a command skeleton, not a fully verified parameter matrix.

From an engineering perspective, --ask's value is that it brings the "model as function" metaphor down to earth. When output is restricted to enumerations or numbers, the model can be orchestrated into shell scripts, CI pipelines, or an agent's decision nodes just like an API, without needing an extra parsing layer.

The Capability Boundaries of a 9B Local Model

The original material explicitly states: this is a 9B-class local model, and "where a local 9B falls short" is one of the questions it's meant to answer.

9B parameters sits in the sweet spot for local inference—consumer GPUs, unified-memory Macs, even some strong CPUs can run it. But parameter count determines its ceiling for judgment:

• Fine-grained domain label classification, especially when categories are semantically close, is prone to confusion
• Judgments requiring external knowledge or time-sensitive information can only rely on priors
• Probability calibration is usually worse than larger models; scores are referenceable but shouldn't be treated as precise probabilities

This also explains why decision models are better suited as a "first filter." Let them handle high-confidence samples and pass ambiguous cases to larger models or humans—this tiered architecture is more cost-effective than simply stacking a single large model.

Nimble's positioning isn't to replace GPT-level judgment, but to provide a good-enough judge under the premise of being local, low-latency, and offline-available. For privacy-sensitive or network-constrained scenarios, this trade-off has practical significance.

Probability Calibration: This refers to whether a model's output probability values truly reflect the frequency of events occurring. For example, when the model gives 0.9 confidence, ideally such samples should be about 90% positive. Small models, due to limited training data scale and fine-tuning objectives that are often cross-entropy rather than Brier score, tend to exhibit "overconfidence"—outputting 0.95 for something that's really a 0.7 certainty. For decision models, this means scores are better suited for ranking (which is more likely to hit) than threshold judgment (anything above 0.8 gets approved).

Common engineering mitigations include temperature scaling, Platt scaling, or isotonic regression, all of which require a batch of labeled validation data to fit. The original material doesn't mention whether Nimble has built-in calibration, so before actually integrating it, it's best to first plot a reliability diagram using known samples to confirm whether its probability distribution is usable. If calibration is poor, treat the output as a three-tier label: high confidence, low confidence, refuse to judge.

What Decision Models Mean for Local AI Workflows

Placing this Ollama update in a larger picture, it points to a trend: the criteria for evaluating model capability are diverging.

In the past, everyone compared MMLU, HumanEval, and other comprehensive benchmarks. But a model that can write poetry and code isn't necessarily able to stably output a correctly formatted yes/no. The latter's difficulty isn't in knowledge, but in output constraints.

The value of decision models is that they build constraints into the model itself, rather than relying on prompt engineering to patch things up afterward. This is especially important for agent systems—if every step of an agent's decision-making requires parsing free text, the error probability accumulates with the number of steps.

For locally deployed users, Nimble provides a new building block: not for chatting, but for serving as a judgment node. It can be embedded into high-frequency but low-complexity tasks like log filtering, email classification, and form validation, saving on cloud calls.

The 9B limitations are real, but the positioning is clear. Knowing what a model isn't suited for is just as important as knowing what it is.

---

Quick Start:

**"Model as Function" and Agent Decision Nodes**: In agent architecture, a tool call or state transition is often abstracted as a function `decision(state) -> action`. If this function is implemented by a chat model, you need extra prompt templates, regex parsing, and failure retries—every additional step in the chain is another chance for error. After replacing the decision node with a restricted-output model like Nimble, the interface becomes the enumeration or score returned by `nimble --ask`, and scripts can directly `case`-branch or compare thresholds without parsing free text.

Another benefit of this approach is that it's **cacheable and auditable**: the same input yields stable output, making it easy to write unit tests and record decision logs. The limitation is that it can only do single-step classification and can't handle planning tasks requiring multi-hop reasoning. So a more reasonable division of labor is: use large models for planning and explanation, use decision models for high-frequency, low-complexity gating and labeling, with the two connected through structured interfaces.
# Pull and run
ollama run nimble --ask

# Or ask directly
ollama run nimble --ask "Is this text in English?"

It's recommended to first test its judgment accuracy and probability distribution using a batch of samples with known answers before deciding whether to integrate it into a production pipeline.