vLLM v0.30.1rc0 Released: Adds AMD MI355 NVFP4 and MoRI Kernel Support

vLLM, one of the most popular LLM inference and serving frameworks available today, has released v0.30.1rc0 as a pre-release candidate. This update focuses squarely on the AMD ROCm ecosystem, introducing dense NVFP4 kernels and MoRI kernel mirror support for the MI355 accelerator (PR #58281). While it's a release candidate rather than a final stable release, the commit history clearly reflects vLLM's ongoing investment in expanding hardware compatibility — particularly on AMD platforms.

What's in This Release

The core commit for v0.30.1rc0 was submitted by AMD engineer Andreas Karatzas, with the title [ROCm][CI] Add MI355 dense NVFP4 and MoRI kernel mirrors. Breaking that down:

• ROCm: AMD's open-source GPU compute software stack, analogous to NVIDIA's CUDA. This update falls under the ROCm CI (continuous integration) path.
• MI355: A high-end AMD Instinct series accelerator designed for data center AI training and inference, offering strong compute throughput and memory bandwidth.
• NVFP4: A 4-bit floating-point quantization format that dramatically reduces memory footprint while preserving model accuracy and boosting inference throughput.
• MoRI kernel: Compute kernel mirrors related to Mixture-of-Experts (MoE) routing operations.

In short, this update fills in the kernel infrastructure AMD MI355 needs for low-bit quantized inference, enabling developers to run NVFP4-quantized models on AMD hardware.

ROCm (Radeon Open Compute) is AMD's full-stack open-source GPU compute platform, encompassing drivers, runtimes, the HIP compiler, and math libraries (rocBLAS, rocSPARSE, etc.). HIP (Heterogeneous-compute Interface for Portability) is ROCm's core programming interface — its syntax closely mirrors CUDA, making it relatively straightforward to port CUDA code to AMD GPUs. Unlike NVIDIA's proprietary CUDA ecosystem, ROCm is fully open source, providing the foundation for third-party frameworks like PyTorch and vLLM to build high-performance kernels on AMD hardware. That said, the ROCm ecosystem has historically lagged behind CUDA in maturity — many kernels optimized for NVIDIA hardware need to be separately reimplemented or "mirrored" for the ROCm path. That's exactly what "kernel mirrors" refers to in this update's title: bringing AMD-side compute kernels up to functional parity with their NVIDIA counterparts.

Why NVFP4 Quantization Matters

NVFP4 represents one of the most important directions in inference optimization in recent years. Compared to FP16 or FP8, a 4-bit floating-point format compresses model weight storage to the absolute minimum — a critical advantage when running models with tens or hundreds of billions of parameters.

For inference serving, quantization delivers a two-pronged benefit: lower memory usage means a single GPU can host larger models or longer contexts, while reduced memory bandwidth pressure typically translates into higher token generation throughput. By adding dense NVFP4 kernels for the MI355, vLLM gives AMD users hardware-level quantized inference capabilities comparable to what's available on NVIDIA platforms. This has real implications for breaking CUDA's ecosystem dominance and broadening the hardware options available for AI inference.

NVFP4 is a 4-bit floating-point format introduced by NVIDIA with the Blackwell architecture (GB200/B200, etc.). It uses a 1-bit sign, 2-bit exponent, and 1-bit mantissa encoding (E2M1) — cutting per-weight storage in half again compared to FP8 (E4M3/E5M2). Compared to integer quantization (INT4), floating-point formats offer more controlled precision loss for small values, making them better suited to preserving the distribution of neural network weights. In practice, NVFP4 is typically used with "dense quantization" — applying uniform quantization across all linear layer weights — as opposed to mixed-precision schemes that only quantize select layers. Notably, NVFP4 is a NVIDIA-originated format; AMD's support for it on the MI355 means AMD has added native hardware compute support for this numeric format, ensuring compatibility with models quantized using it. This is a significant step for cross-platform interoperability.

AMD's Continued Push

One notable signal in this release: the commit was led by an AMD official engineer (with an @amd.com email address), and the commit message includes a Co-authored-by: OpenAI Codex attribution. This reflects a broader trend — AMD platform support in vLLM is increasingly being driven directly by the chipmaker itself, rather than relying on community volunteers.

For years, AI inference framework optimization has centered on the NVIDIA CUDA ecosystem. By directly contributing MI-series accelerator kernel support to mainstream open-source projects like vLLM, AMD is executing on its strategy to grow market share in AI inference. For enterprise users looking to reduce hardware costs or diversify their supply chain, a mature AMD inference path is steadily taking shape.

AMD Instinct MI355X is the latest generation of AMD's CDNA3 architecture data center AI accelerators, featuring 288GB of HBM3E memory and a peak FP8 compute throughput of 2.6 PFLOPS — surpassing NVIDIA's H100 (80GB) and H200 (141GB) in memory capacity, making it AMD's flagship product for ultra-large-scale model inference and training. AMD positions the MI300 series (including the MI355) as a direct competitor to NVIDIA's H100/H200 and B200. However, beyond raw hardware performance, software ecosystem completeness is often the deciding factor for enterprise adoption — whether an inference framework has kernel-level support for specific quantization formats directly determines whether new hardware can fully deliver its compute potential in production. This is precisely why AMD engineers are contributing kernel code directly to vLLM rather than waiting for the community to organically develop support.

A Note on Release Candidates

It's worth noting that v0.30.1rc0 is a release candidate (rc = release candidate), not a final stable release. These builds are typically used to gather feedback and validate stability before an official release. Production users are advised to wait for the final version, while developers eager to test MI355 NVFP4 support can use this build for evaluation and validation.

With over 92,000 GitHub Stars and 22,600 Forks, vLLM's fast iteration pace and broad hardware coverage have cemented its status as infrastructure-grade tooling for LLM inference. This update targeting AMD MI355 is yet another expansion of its hardware ecosystem footprint.

Takeaways

While v0.30.1rc0 is a relatively modest candidate release, its introduction of dense NVFP4 and MoRI kernels for AMD MI355 signals two clear trends: low-bit quantized inference is becoming a standard expectation, and AMD is actively accelerating the build-out of an AI inference ecosystem capable of competing with NVIDIA — doing so through direct contributions to mainstream open-source frameworks. For teams focused on inference performance optimization and hardware diversification, this is an update worth watching.