Decision 3.0 Open-Sourced: 0.6B–27B Multimodal Decision Models Top Both Leaderboards

The vLLM Semantic Router team has released Decision 3.0 (d3), a new family of decision models spanning parameter scales from 0.6B to 27B. Licensed under Apache-2.0, the models support three input modalities — text, images, and video. The release has taken the top spot on both the text and vision tracks of the Jev Decision Index, establishing a new SOTA in the decision model space.

A Model Family Built for Decisions, Not Generation

Unlike the generative large models we're used to, Decision 3.0 has a very clear purpose — it doesn't generate text. Instead, it focuses exclusively on outputting decision results. The models support three decision types: multiple choice (choice), yes/no judgment, and scoring (score), and they output a probability for each candidate answer rather than free-form text.

This design introduces a noteworthy engineering characteristic: multiple questions about the same input can be processed in a single call, with each question performing one forward pass over the input. For scenarios that require answering multiple structured questions about the same piece of content, this approach offers clear advantages in both efficiency and controllability. Compared to using a large language model to generate answers and then parse them, directly outputting probabilities is far better suited for integration into downstream automated decision pipelines.

Models focused on decision outputs like these are generally known in the industry as "Discriminative Models" or "Classification Models," in contrast to "Generative Models" like GPT and Llama. Generative models produce free text through token-by-token sampling, while discriminative models directly compute posterior probabilities over a predefined set of candidates. In use cases such as routing, content moderation, and relevance ranking, the output of a discriminative model is naturally a comparable numerical value — no additional parsing needed, and latency is typically lower. The vLLM Semantic Router team has branded these models "Decision Models," emphasizing their role as decision nodes in automated pipelines rather than information generators.

Underlying Architecture and Multimodal Capabilities

Decision 3.0 is built on the Qwen series — specifically Qwen3.5 (0.8B / 2B / 4B / 9B) and Qwen3.8-27B. The team explicitly notes that the published parameter counts include the visual encoder — for example, the visual encoder accounts for 0.33B in the 4B version and 0.46B in the 27B version. This means these figures reflect the total multimodal system scale, not just the language backbone.

For multimodal inputs, the model's processing capabilities are quite specific:

• Images: Multiple images per request, with a maximum resolution of 1.6 MP.
• Video: Multiple video files per request, supporting formats including MP4, WebM, MOV, and MKV. Sampling is performed at 2 fps, with a maximum of 32 frames per video and a per-frame resolution cap of 0.2 MP.

Model weights are in BF16 precision and can be loaded via the Transformers library (requires setting trust_remote_code=True). Training was completed on AMD Instinct MI325X accelerators — a choice that reflects the growing diversification of the AI training hardware ecosystem.

BF16 (Brain Float 16) is a 16-bit floating-point format developed by Google. Compared to the IEEE standard FP16, BF16 retains the same 8-bit exponent as FP32 while compressing only the mantissa. This design gives BF16 the same numerical range as FP32, greatly reducing the risk of overflow or underflow during training, which is why it is widely used in large model training and inference. The AMD Instinct MI325X is AMD's data center AI training and inference accelerator, based on the CDNA 3 architecture with support for BF16 and FP8 computation. The team's decision to train on AMD hardware and disclose this publicly provides useful reference for evaluating reproducibility and deployment costs outside the NVIDIA ecosystem, and serves as a precedent for developers using the ROCm software stack.

Benchmark Results: Leading Across All Parameter Tiers

Based on the team's evaluation using the official Jev 0.3 benchmark suite on the released weights, Decision 3.0 delivers strong results.

The 27B version scored 64.7 on the text leaderboard and 71.6 on the vision leaderboard, outperforming all competitors — including Jev's own models — to claim first place on both.

Perhaps more practically informative is the tiered performance across sizes: the 0.6B, 0.8B, 4B, and 9B versions each lead their respective parameter-scale tiers. Notably, the 9B version (codenamed d3-flash) even surpasses the previous-generation 27B model (Decision 2.0), suggesting that improvements in training methodology or data have enabled smaller models to exceed the capabilities of the prior flagship. This is good news for deployments seeking a balance between inference cost and performance.

Unexpectedly Strong Zero-Shot Video Understanding

One of the most striking results is the models' zero-shot video capability. The team explicitly states that none of the models were trained on any video data. Yet on all 19,140 questions of the Perception Test validation set, the 27B version scored 77.4 and the 4B version scored 70.3.

The fact that the models can transfer meaningful video understanding ability — via frame-by-frame sampling — without ever having seen video training data demonstrates strong generalization of their image modality capabilities to temporal visual content. For developers looking to cover both image-text and video decision tasks with a single model, this zero-shot generalization lowers the barrier to entry by removing the need to collect additional video training data.

Zero-shot generalization refers to a model's ability to perform a task it has never seen training examples for, using only its existing knowledge. In the multimodal domain, video understanding is generally considered harder than static image understanding because it additionally involves temporal relationships and cross-frame consistency. Decision 3.0 addresses this by "flattening" video into a sequence of static images — sampling at 2 fps and taking at most 32 frames — effectively converting video questions into multi-image input problems. Perception Test is a multi-task video understanding benchmark published by DeepMind, covering 19,140 questions across perception, memory, and abstract reasoning, and is widely regarded as an authoritative dataset for evaluating video understanding. Achieving a zero-shot score of 77.4 (27B) on this benchmark suggests that frame-level image understanding can, to a meaningful degree, substitute for dedicated temporal modeling.

What This Means for Developers

Several characteristics of Decision 3.0 combine to form a distinctive value proposition: the permissive Apache-2.0 license allows commercial use; the wide range of parameter scales makes it easy to choose the right fit; the design of outputting probabilities directly rather than generating text is a natural fit for structured decision tasks; and multimodal support with zero-shot video generalization broadens the range of applicable use cases.

The complete lineup from 0.6B to 27B means there's an appropriate option for every compute environment, from edge devices to data centers. Model weights are publicly available on Hugging Face (vllm-sr/decision-30 collection) for developers who want to evaluate and integrate them directly.

One caveat worth noting: the benchmark results reported above come from the team's own evaluation using the official benchmark suite. Independent third-party replication and real-world testing across a broader range of scenarios will remain the critical next step in validating these capabilities.