AMD MI355X
The AMD MI355X is a high-end AI accelerator GPU in AMD's Instinct series for data centers, designed for large-scale artificial intelligence inference and training workloads. Built on AMD's advanced GPU architecture, it features large-capacity high-bandwidth memory (HBM) and supports high-performance matrix operations. It is suited for large language model inference, scientific computing, and cloud-based AI deployment, and represents one of AMD's flagship compute accelerator products for enterprise-grade AI infrastructure.
Timeline (last 90 days)
For buyers, a near-doubling gap in tokens per dollar is enough to change TCO calculations; for developers, the combination of ROCm and vLLM is beginning to reach production readiness.
Under the same benchmark conditions, the per-GPU throughput of the 8-GPU AMD MI355X is 24.4% higher than the 6,083 tok/s/GPU of the HGX B300.
Calculated as tokens processed per dollar, the advantage of the 8-GPU AMD MI355X widens to 87.4%.
This performance improvement is mainly attributed to three kinds of work: scheduling and memory management optimizations at the vLLM level, kernel implementations customized for the AMD architecture, and fine-tuning of hardware-level details.
The MI355X this time matched or even surpassed the B300 in real agentic workloads, indicating that AMD's weakness is shifting from 'the hardware isn't good enough' to 'the software needs improvement.'
In SemiAnalysis's AgentX benchmark, an 8-GPU AMD MI355X achieved 7,567 tok/s per GPU in Kimi K3 model serving at an interactive latency of 50 tok/s/user.
Meta provides two dedicated draft models for speculative decoding: DFlash V2 is designed for SGLang and is reportedly achieving a 3.7x throughput improvement on AMD MI355X hardware, accepting an average of 6.4 draft tokens per step; DSpark is designed for vLLM and proposes up to seven tokens per pass.
GLM-5.3-Flash can now run on AMD MI355X and supports FP8/MXFP4 MoE and MTP speculative decoding.
On MI355X, by removing unnecessary expert padding, the FP4 MoE model weights of DeepSeek-V4 were reduced from 159.07 GB to 112.36 GB.
Kimi K3 on AMD MI355X using grouped-head MLA verification kernels achieves 1.37–1.77× throughput improvement and 1.45–2.42× ITL improvement at concurrencies of 2–32.
All Facts (10)
For buyers, a near-doubling gap in tokens per dollar is enough to change TCO calculations; for developers, the combination of ROCm and vLLM is beginning to reach production readiness.
50%UnverifiedUnder the same benchmark conditions, the per-GPU throughput of the 8-GPU AMD MI355X is 24.4% higher than the 6,083 tok/s/GPU of the HGX B300.
50%UnverifiedCalculated as tokens processed per dollar, the advantage of the 8-GPU AMD MI355X widens to 87.4%.
50%UnverifiedThis performance improvement is mainly attributed to three kinds of work: scheduling and memory management optimizations at the vLLM level, kernel implementations customized for the AMD architecture, and fine-tuning of hardware-level details.
50%UnverifiedThe MI355X this time matched or even surpassed the B300 in real agentic workloads, indicating that AMD's weakness is shifting from 'the hardware isn't good enough' to 'the software needs improvement.'
50%UnverifiedIn SemiAnalysis's AgentX benchmark, an 8-GPU AMD MI355X achieved 7,567 tok/s per GPU in Kimi K3 model serving at an interactive latency of 50 tok/s/user.
50%UnverifiedMeta provides two dedicated draft models for speculative decoding: DFlash V2 is designed for SGLang and is reportedly achieving a 3.7x throughput improvement on AMD MI355X hardware, accepting an average of 6.4 draft tokens per step; DSpark is designed for vLLM and proposes up to seven tokens per pass.
50%UnverifiedGLM-5.3-Flash can now run on AMD MI355X and supports FP8/MXFP4 MoE and MTP speculative decoding.
50%UnverifiedOn MI355X, by removing unnecessary expert padding, the FP4 MoE model weights of DeepSeek-V4 were reduced from 159.07 GB to 112.36 GB.
50%UnverifiedKimi K3 on AMD MI355X using grouped-head MLA verification kernels achieves 1.37–1.77× throughput improvement and 1.45–2.42× ITL improvement at concurrencies of 2–32.
50%