AI Decision Models: When Your AI Should Return Probabilities, Not Paragraphs

Most teams handle a customer support ticket in almost the same way: they throw the ticket at a chat model and use a prompt to force it to output the classification result in JSON. The model generates dozens of tokens of JSON, you parse it, and occasionally retry because some field has a trailing space. This pipeline works, but the cost is ignored—you used a text generator to make three small decisions, paid for each output token, waited for them to stream out one by one, and ended up with a label while having no idea how certain the model was.

Liquid AI recently released two open-weight models built around a different philosophy, called decision models. Even if you don't use these two specific models, it's worth understanding the thinking behind them, because it changes how you think about the large number of LLM calls in your product.

What Liquid AI released

According to Liquid AI's official blog and Hugging Face release post, its d1 series includes two open-weight models:

• d1-3B (about 3.1 billion parameters), trained on the LFM2.5-VL-3B vision-language backbone, supporting text and image inputs.
• d1-omni-600M (about 587 million parameters), based on a small bidirectional encoder, with added vision and audio encoders. Supports text plus images, or text plus audio up to 30 seconds. Liquid calls it an early research release.

The key is not what they can do, but what they don't do. Neither model generates text. You give the model an input (called a "state") and a set of named questions, and it returns typed answers with probabilities in a single forward pass: yes/no answers with probabilities, a choice among options you define with corresponding probabilities, or a score on a scale you describe. MarkTechPost's coverage summarizes it as "zero output tokens," which is the most concise way to put it.

Why a single forward pass matters

Chat model latency mainly comes from output tokens: generate one token, feed it back, generate the next. Even a short JSON answer contains dozens of serial steps. A decision model, by contrast, reads the input once and reads the answer directly from the final layer. That's it.

According to d1-3B data provided by Liquid in the Hugging Face release post: on an NVIDIA RTX 4090, answering one question takes just 8 ms, and three questions take 21 ms; on Jetson AGX Thor it's 16 ms and 20 ms respectively; on Jetson AGX Orin 64GB it's 26 ms and 35 ms; and on Jetson Orin Nano it's 50 ms and 73 ms.

On quality, the official report gives an average score of 82.9 across seven public text benchmarks (reading comprehension, toxicity detection, intent classification, medical QA, and cross-lingual tasks), and 48.57 on the public split of its Decision Index v0.2.1. These are currently vendor numbers and need independent evaluation to verify. ModelSystem.One pointed out that the official index snapshot it examined predates these model cards. Even so, the core significance of the results is clear: a 3B model answers in milliseconds on an edge board, with no output tokens to pay for or wait on.

Forward pass refers to the process in which a neural network performs a complete computation once from the input layer to the output layer to produce a result, as opposed to backpropagation during training. An autoregressive language model generating N tokens actually performs N forward passes: each time it rereads the entire input and already-generated tokens, computes the probability distribution for the next token, samples from it, and then appends the new token back to the input to repeat. This causes latency to grow linearly with output length, and it is exactly the source of token-by-token streaming output. A classifier-style model, however, needs only one forward pass: the input is read once, and the answer distribution is read directly from the final layer, with no loop, no sampling, and no writing output back into the input. This is the fundamental reason decision models can achieve millisecond latency—their computation scales linearly with input length and has nothing to do with "output length," because there is no output sequence to generate at all.

The "state" in a decision model refers to the one-shot input representation the model receives, which can be the encoding of a piece of text, the embedding of an image, or the encoding of audio features, depending on the backbone model. Unlike a chat model that concatenates conversation history into a prompt, the state is usually a fixed-dimensional vector or tensor produced by a single compression pass through an encoder. After that, each "question" performs one lightweight computation against the same state (such as a classification head or a small scoring module). That is why three questions take 21 ms rather than three times the 8 ms of one question—the state encoding is done only once, and the marginal cost of asking one more question is very low. Understanding this helps with usage design: packaging all decisions on the same input into one set of questions and sharing a single encoding is the most economical way to use this architecture.

What it looks like in code

The official example shows the usage pattern. You define questions as data structures rather than prompts asking for JSON:

from transformers import AutoModel

model = AutoModel.from_pretrained("LiquidAI/d1-3B", trust_remote_code=True)

questions = {
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
    "team": {"type": "choice", "instructions": "Which team should handle this?",
             "criteria": {"billing": "Charges, refunds, invoices",
                          "technical": "App or site faults",
                          "fraud": "Suspected unauthorised use"}},
    "urgency": {"type": "score", "instructions": "How urgent is this?",
                "criteria": ["Can wait", "Today", "Blocking the customer now"]},
}

print(model.system_one("I was charged twice this month, please refund one of them.", questions))

Notice what disappears: no "reply only in JSON," no parsing, no retry loop for malformed output. The output space is fixed by the questions, and the model cannot invent a fourth team out of thin air. It requires transformers>=5.14 and trust_remote_code=True, and you should review the model code before running it in production.

The real advantage: probabilities you can act on

A chat model says "team": "billing". A decision model roughly says "billing 0.91, fraud 0.07, technical 0.02." That number is the most useful part, because it lets you build a confidence router:

def route(ticket, decide, escalate):
    result = decide(ticket)              # fast decision model, millisecond-level
    label, p = result.top()
    if p >= 0.90:
        return label                     # execute automatically
    if p >= 0.60:
        return f"{label} (needs review)" # queue for a human or a larger model
    return escalate(ticket)              # hard cases: hand off to a full LLM

This is the pattern truly worth taking away. Most production AI systems have a small number of these small-scale, high-frequency decisions: whether something is toxic, what the intent is, whether the retrieved passage answers the question, whether an agent step should continue, whether an image has defects. Run them on a small, fast model, act directly on high-confidence cases, and spend large-model budget only on the uncertain long tail.

An honest caveat: "calibration" means probabilities should match reality (something it thinks is 0.9 should be about 90% correct). Don't blindly trust it. Before setting thresholds, check it with your own labeled data, just as you would with any classifier.

Calibration is the key property determining whether probability outputs can be trusted. In a well-calibrated model, when it outputs a probability of 0.9, about 90% of such samples should truly belong to that class. But neural networks generally tend to be overconfident: high probabilities output by softmax do not necessarily reflect true frequencies, especially on inputs outside the training distribution, where the model may give 0.99 and still be completely wrong. This is why the article emphasizes not blindly setting thresholds. A practical approach is: collect a batch of manually labeled samples, plot a reliability diagram (bucketing predicted probabilities vs. actual accuracy), or compute metrics such as ECE, and use that to decide where to set the "automatic execution" threshold. If calibration is poor, consider post-processing methods such as temperature scaling, or simply use thresholds plus manual spot checks as a fallback. The probabilities from a decision model only truly have routing value after calibration has been verified.

When it fits and when it doesn't

Good fits:

• Moderation and safety filtering for every message
• Intent and ticket routing
• RAG relevance checks ("Does this passage answer the question?")
• Guardrails inside an agent loop ("Is this tool call safe?")
• On-device or edge checks with no network budget: visual inspection, robotics, self-service kiosks

Not good fits:

• Tasks where the answer needs open-ended text: summaries, drafts, explanations
• Decisions where the options are not known in advance
• Scenarios requiring long, multi-step reasoning

Practical cautions:

• The open models use the LFM Open License v1.0, and according to MarkTechPost, commercial use is free under $10 million in annual revenue. Read the license yourself before release.
• d1-omni-600M is explicitly an early research release, with no published latency data.
• The benchmarks are self-reported, and audio decision benchmarks remain an open problem in Liquid's view.

A confidence router is a common architectural pattern for AI systems: use a small, fast model for front-line judgments, and use its output probabilities to decide whether to execute automatically, send for human review, or escalate to a larger model. Its economics come from the long-tail nature of the distribution—most inputs are actually simple and common, and a small model can handle them correctly with high confidence, while only a few hard samples require an expensive large model. This design concentrates large-model budget where it is truly needed, while explicitly controlling the trade-off between accuracy and cost through probability thresholds. The thresholds themselves are not fixed and should be adjusted based on calibration results and business tolerance: safety-sensitive scenarios can raise the automatic-execution threshold, while throughput-focused scenarios can lower it.

The bigger picture

The industry spent two years routing every AI problem to a chat interface because frontier models were like that. But many of what we call "LLM features" are really just classifiers wearing chat clothes. When the output is a decision, a model that only makes decisions is faster, cheaper, easier to verify, and tells you how certain it is. You don't need to switch to d1 tomorrow, but it's worth spending an hour listing every LLM call in your product whose output is essentially a label, a yes/no, or a score. That list is where your latency and cost savings are hiding.

References

• Liquid AI official blog: Open d1, edge decision models for text, vision, and audio
• Hugging Face blog: Multimodal open d1 decision models for the edge
• Liquid AI blog: Introducing d1, the hosted decision model with vision
• MarkTechPost: Liquid AI releases open-weight d1-3B and d1-omni-600M
• ModelSystem.One: Liquid AI ships open d1-3B and d1-omni-600M

How many LLM calls in your product are actually just a label, a yes/no, or a score?

One-line Summary

Liquid AI releases open models d1 that return probabilistic decisions in a single forward pass without generating text.

Paragraph Summary

Liquid AI released two open-weight decision models, d1-3B and d1-omni-600M, whose core feature is making decisions without generating text: given an input state and a set of named questions, the model returns typed answers with probabilities (yes/no, choices, scores) in a single forward pass, achieving zero output tokens. This lets it answer one question in just 8 ms on an RTX 4090 and three questions in 21 ms, making it suitable for edge devices and high-frequency small-decision scenarios. The article notes that probability outputs are the greatest value, enabling a confidence router: execute automatically at high confidence, hand off to humans or larger models at medium confidence, and escalate at low confidence. It fits moderation filtering, intent routing, RAG relevance checks, and agent guardrails, but not open-ended text generation. The author cautions that probabilities need calibration verification.