Model Runner V2
The second-generation model execution engine for the vLLM framework, featuring a comprehensive redesign of the underlying scheduling logic, memory management, and batching strategy. It became the default runtime for all models starting with vLLM v0.29.0.
Timeline (last 90 days)
vLLM released release candidate v0.31.0rc3, with the core highlight being the introduction of Randomized Dummy Inputs support for Model Runner V2.
v0.30.0 launches a new Model Runner V2, bringing EAGLE3-style draft generation into pipeline parallelism.
In vLLM v0.29.0, Model Runner V2 officially became the default execution engine for all models.
Teams using vLLM in production environments are advised to evaluate potential behavioral changes caused by the default engine switch before upgrading, and to validate throughput and latency metrics in a test environment.
Model Runner V2 gains two capabilities in vLLM v0.28.0: E/P/D separation (Encode/Prefill/Decode disaggregated deployment) and weight offloading.
Model Runner V2 (MRV2) capabilities are extended to non-generative workloads, supporting encoder-only attention, sequence pooling, BGE-M3 pooling, and multimodal execution on CPU.
All Facts (6)
v0.30.0 launches a new Model Runner V2, bringing EAGLE3-style draft generation into pipeline parallelism.
50%UnverifiedTeams using vLLM in production environments are advised to evaluate potential behavioral changes caused by the default engine switch before upgrading, and to validate throughput and latency metrics in a test environment.
50%UnverifiedModel Runner V2 gains two capabilities in vLLM v0.28.0: E/P/D separation (Encode/Prefill/Decode disaggregated deployment) and weight offloading.
50%UnverifiedModel Runner V2 (MRV2) capabilities are extended to non-generative workloads, supporting encoder-only attention, sequence pooling, BGE-M3 pooling, and multimodal execution on CPU.
50%UnverifiedvLLM released release candidate v0.31.0rc3, with the core highlight being the introduction of Randomized Dummy Inputs support for Model Runner V2.
50%UnverifiedIn vLLM v0.29.0, Model Runner V2 officially became the default execution engine for all models.
50%