GB300 NVL72
A rack-scale system from NVIDIA for large-scale AI inference. It integrates dozens of GPUs into a single logical compute resource through high-speed interconnects, supporting low-latency inference for very large parameter models.
Timeline (last 90 days)
Nvidia CentML engineer Benjamin, through upstream vLLM optimization, enabled the 2.8-trillion-parameter Kimi K3 to achieve an inference speed of 423 tok/s/user on GB300 NVL72, a performance improvement of 1.68 times.
NVIDIA announced that Muse Spark Open has completed adaptation and performance validation on the GB300 NVL72 platform.
GB300 NVL72 is a rack-scale system for large-scale AI inference, designed to support low-latency inference of ultra-large parameter models within a single cluster.
Beam's pre-training used 6,144 NVIDIA GB300 NVL72 chips, took less than 4 weeks, and consumed 23.8 trillion tokens.
All Facts (4)
Nvidia CentML engineer Benjamin, through upstream vLLM optimization, enabled the 2.8-trillion-parameter Kimi K3 to achieve an inference speed of 423 tok/s/user on GB300 NVL72, a performance improvement of 1.68 times.
50%UnverifiedGB300 NVL72 is a rack-scale system for large-scale AI inference, designed to support low-latency inference of ultra-large parameter models within a single cluster.
50%UnverifiedBeam's pre-training used 6,144 NVIDIA GB300 NVL72 chips, took less than 4 weeks, and consumed 23.8 trillion tokens.
50%UnverifiedNVIDIA announced that Muse Spark Open has completed adaptation and performance validation on the GB300 NVL72 platform.
50%