Vela 3.0 Released: Routing and Span Extraction Combined into One

According to technical updates on Twitter, the vLLM Semantic Router, KR Labs, and AMD teams have collaborated to release Vela 3.0. The core change in this update is combining routing decisions and span extraction into a single call, along with six open models ranging from 0.3B to 27B.

From Two Calls to One: An Architectural Merge

In traditional semantic routing systems, routing decisions and span extraction are usually two separate steps: first determine which model or processing pipeline the request should be sent to, then extract key spans from the input. Vela 3.0 combines these two steps into a single call, meaning the system can output the required span information at the same time it decides on routing.

The immediate impact of this merge is reduced call overhead and latency. For applications requiring real-time responses, one less round trip means faster end-to-end response. At the same time, a single call also reduces the risk of information inconsistency between two separate calls.

Semantic Routing refers to the technique of using a lightweight model to assess the semantics of a user request and then decide which downstream model, tool, or processing pipeline to dispatch the request to. Unlike traditional keyword- or rule-based routing, it relies on the model's understanding of intent, so it can maintain stable dispatch results even when wording varies. Semantic routing is typically deployed at the request entry point and is latency-sensitive, so model size and number of calls directly determine its usability.

Span extraction originates from information extraction tasks and refers to locating and extracting a contiguous character interval from input text—for example, identifying the key entities or instruction spans in a user query that actually need to be retrieved. In routing scenarios, span extraction is often used to strip redundant context and construct more precise downstream inputs. In traditional implementations, it belongs to a different model or a different call round than the routing decision. Vela 3.0 merges the two, essentially producing both the classification result and the span localization result in a single forward computation.

Six Open Models Covering 0.3B to 27B

Vela 3.0 provides six open models with parameter ranges from 0.3B to 27B. This span covers a variety of needs, from extremely lightweight deployment to stronger reasoning capability. The 0.3B-level models are suitable for resource-constrained edge environments or high-concurrency scenarios, while the 27B-level models can handle more complex semantic judgment tasks.

The official description emphasizes two points: more flexibility in what the router can detect, and how small it can run. The former points to expanded detection capability, while the latter points to a lower deployment threshold—together, these determine the applicable scope of semantic routing in real production environments.

Open Models generally refer to models whose weights are publicly available and which can be deployed and fine-tuned independently, as opposed to closed-source models that can only be accessed via API. For an entry-point component like semantic routing, open weights mean localized deployment, control over data not leaving the domain, and the ability to distill or fine-tune for specific business corpora. A 0.3B-level model, after quantization, can run on CPUs or low-end accelerators, making it suitable for high-concurrency, low-cost edge filtering; the 27B level is closer to the upper limit of general small-model capability and can handle finer-grained semantic distinctions. The wide model spectrum allows the same routing framework to use models of different sizes during development validation and production deployment while keeping the interface consistent.

Background of the Three-Party Collaboration

This release was completed jointly by the vLLM Semantic Router, KR Labs, and AMD teams. vLLM Semantic Router itself is a semantic routing component around the vLLM ecosystem, and the participation of KR Labs and AMD indicates deeper integration intent at the inference infrastructure level. AMD's involvement is particularly noteworthy, as it may mean Vela 3.0 has been adapted or optimized for AMD hardware platforms.

vLLM is currently a mainstream high-throughput large model inference engine, with PagedAttention and continuous batching as its core optimizations, widely used for model serving in production environments. vLLM Semantic Router is built on the vLLM ecosystem and belongs to the scheduling layer before requests enter the inference engine. AMD's involvement on the inference side usually involves adaptation of the ROCm software stack and Instinct series accelerators. If Vela 3.0 has been optimized for AMD platforms, it means this routing component can achieve usable inference performance in non-NVIDIA GPU environments, which has practical significance for teams looking to reduce hardware lock-in risk.

Significance for Semantic Routing Practice

The core value of semantic routing lies in making sufficiently good dispatch decisions at a lower model cost. Vela 3.0 combines decision and extraction and provides a model spectrum from small to large, effectively lowering the adoption cost of this approach: developers can start with a small 0.3B model to validate results, then upgrade to larger models as needed.

The design of completing routing and span extraction in a single call also simplifies interface complexity during system integration. For teams building multi-model scheduling, intent recognition, or content filtering pipelines, this merged interface is worth attention.

Currently, the officially disclosed information focuses on these two capability points, and specific performance data and benchmark comparisons have not yet been detailed. Future attention can be paid to the measured performance of models of various sizes in terms of routing accuracy and latency.

One-line Summary

Vela 3.0 combines routing decisions and span extraction into a single call and provides six open models from 0.3B to 27B.

Paragraph Summary

The vLLM Semantic Router, KR Labs, and AMD teams jointly released Vela 3.0. The core change is combining routing decisions and span extraction in semantic routing into a single model call, thereby reducing call round-trip overhead, lowering end-to-end latency, and avoiding information inconsistency between two calls. Six open models are also released, with parameter ranges covering 0.3B to 27B, balancing resource-constrained edge high-concurrency scenarios and more complex semantic judgment tasks. The official description emphasizes more flexibility in what the router can detect and a smaller minimum runnable scale, which together expand the applicable scope of semantic routing in production environments. AMD's participation in the three-party collaboration suggests intent to adapt or optimize for AMD hardware platforms. Specific performance data and benchmark comparisons have not yet been disclosed.