Decision 3.0 Released: A Look at the Open-Source Decision Models Behind vLLM Semantic Router

What Is Decision 3.0?

vLLM Semantic Router recently released Decision 3.0 — a family of multimodal decision models licensed under Apache-2.0. The series includes 5 models ranging from 0.8B to 27B parameters, covering deployment scenarios from lightweight edge inference to high-accuracy reasoning.

Unlike traditional large language models, Decision 3.0 is not designed to generate text. Instead, it focuses on making decisions. In a single call, it can process both text and images and answer three types of structured questions: multiple choice (choice), yes/no (yes/no), and scoring (score). This design makes model outputs more deterministic and controllable, and better suited for embedding into automated pipelines.

What Does "Without Generating Text" Actually Mean?

The most notable feature of Decision 3.0 is that it operates without generating text. Conventional LLMs produce full natural language output token by token, which is computationally expensive and prone to formatting inconsistencies and hallucinations.

Decision models reframe the task as a classification or scoring problem. When you need to know "does this image contain prohibited content?" (yes/no), "which category does this text belong to?" (choice), or "what's the quality score of this response?" (score), the model can return a structured result directly — no need to generate a descriptive passage first.

This paradigm is especially well-suited for routing, content moderation, and quality assessment workflows. vLLM Semantic Router itself is a semantic routing component, and decision models serve as its core engine — quickly determining which downstream model or service a request should be directed to.

From a technical standpoint, these models typically skip the autoregressive decoding process at inference time, instead attaching a classification head or regression head directly to the final hidden states. This is conceptually similar to fine-tuning BERT-style models for classification, but Decision 3.0 takes it further with alignment training on top of a vision-language pretrained model, enabling joint understanding of image and text semantics before producing a judgment. The direct engineering benefit is extremely low latency — by eliminating the token-by-token generation loop, inference time complexity drops from O(n) to near-constant, delivering a clear throughput advantage in high-concurrency routing scenarios.

Engineering Considerations for Multimodal, Multi-Scale Design

A key highlight of Decision 3.0 is its support for multimodal input — text and images can be processed together in a single call. For applications requiring cross-modal judgment — such as image-text consistency checks or content rating with accompanying visuals — this eliminates the need to chain multiple single-modal models, significantly simplifying the inference pipeline.

The graduated model size range from 0.8B to 27B also reflects a deliberate engineering philosophy. Smaller models (e.g., 0.8B) are suited for edge devices or real-time routing scenarios with strict latency requirements, while larger models (27B) can handle tasks where higher judgment accuracy matters. Developers can choose the right version based on their own accuracy-vs-cost tradeoffs.

Semantic Routing refers to the mechanism of dispatching requests to the most appropriate downstream processing node based on the semantic content of the request — similar to load balancing at the network layer, but based on the meaning of natural language or multimodal content rather than fixed rules. vLLM Semantic Router is a concrete implementation of this pattern: it places a lightweight routing layer in front of the vLLM inference engine, using a decision model to quickly classify the request type and forward it to a specialized model optimized for that type. This architecture can significantly reduce average inference cost without sacrificing overall accuracy — high-frequency simple requests go to small models, while complex requests invoke large ones, creating a dynamic balance between cost and quality.

What the Open-Source License Means

The entire Decision 3.0 series is open-sourced under the Apache-2.0 license, meaning enterprises and developers can integrate, modify, and redistribute these models in commercial products relatively freely, without worrying about restrictive licensing constraints. For teams looking to build their own semantic routing or decision pipelines without relying on closed-source APIs, this lowers both the technical barrier and compliance risk.

Combined with the vLLM ecosystem — a high-performance inference framework — the release of Decision 3.0 further enriches the open-source AI toolchain. The "decision model" paradigm it represents may prompt more teams to reconsider a fundamental assumption: not every task requires a large model that "talks." Sometimes, a fast and deterministic judgment engine is simply a better fit.

Summary

Decision 3.0 positions itself around four core strengths: open-source, multimodal, multi-scale, and non-generative decision-making. It offers a new set of tools for semantic routing and structured judgment tasks. While publicly available technical details remain limited, its product positioning is clear, and it has strong potential for real-world adoption in content moderation, intelligent routing, and quality evaluation scenarios. Developers interested in the project should keep an eye on its upcoming integration progress within the vLLM ecosystem.