InfiniBand
A network communication standard for high-performance computing, characterized by microsecond-level ultra-low latency and bandwidths exceeding 400 Gb/s. Widely deployed in supercomputing centers and large-scale AI training clusters for node interconnection, it is one of the mainstream high-performance interconnect solutions.
Core Facts
NVLink 4.0 offers a bidirectional bandwidth of up to 900 GB/s, while cross-node InfiniBand HDR offers only approximately 50 GB/s, a gap of nearly 20 times.
AI training typically requires thousands of high-end GPUs working in concert for weeks or even months, with extremely high demands on bandwidth, latency, and computational consistency, making it currently almost exclusively feasible in centralized data centers.
Timeline (last 90 days)
RoCE和InfiniBand是常见的RDMA实现。
InfiniBand是一种高吞吐、低延迟的网络互连标准，广泛应用于超算和AI训练集群。
AMD's ROCm ecosystem achieves RDMA capability by supporting RDMA over Converged Ethernet (RoCE), competing with InfiniBand-based solutions.
GPUs transmit gradient data via InfiniBand or RoCE protocols, with bandwidth reaching 400 Gbps per card or higher.
Tensor parallelism is typically used only between NVLink-connected GPUs within the same node; cross-node deployment severely slows training throughput due to insufficient InfiniBand bandwidth
RoCE is an implementation of RDMA over standard Ethernet, with lower deployment costs compared to traditional RDMA deployments that require dedicated InfiniBand networks.
NVIDIA's InfiniBand network and RoCEv2-based solutions have become the mainstream choices for large-scale AI training clusters.
When multiple GPUs are interconnected via NVLink or InfiniBand, the interconnect bandwidth often becomes the hard ceiling for end-to-end inference throughput rather than GPU compute itself.
In traditional networking solutions, network jitter and protocol overhead typically result in actual bandwidth utilization of only 60%–70%
Traditional GPU servers need to transfer data across hundreds to thousands of chips via high-speed interconnect protocols such as NVLink and InfiniBand, which consumes significant bandwidth and introduces latency and additional power consumption.
13 more timeline events
All Facts (20)
NVLink 4.0 offers a bidirectional bandwidth of up to 900 GB/s, while cross-node InfiniBand HDR offers only approximately 50 GB/s, a gap of nearly 20 times.
80%VerifiedAI training typically requires thousands of high-end GPUs working in concert for weeks or even months, with extremely high demands on bandwidth, latency, and computational consistency, making it currently almost exclusively feasible in centralized data centers.
65%Unverified传统DGX集群中，8块GPU组成一个节点，节点内走NVLink，节点间走InfiniBand
90%Unverified当前主流的NDR InfiniBand提供400Gbps的端口速率
90%Unverified大语言模型训练背后需要精密的数据中心电源分配单元、液冷或风冷散热系统、高速互联网络（NVLink、InfiniBand）以及UPS不间断电源保障
90%Unverified在大规模训练集群中，节点内通信（NVLink）与节点间通信（InfiniBand或RoCE）的性能差异可达一个数量级
80%UnverifiedGPU interconnect bandwidth within data centers is typically on the order of 100–400 Gbps (via InfiniBand HDR/NDR or NVSwitch), while wide-area network bandwidth between data centers may be only 1–10 Gbps.
70%UnverifiedNVIDIA acquired InfiniBand technology through its acquisition of Mellanox.
70%UnverifiedRoCE和InfiniBand是常见的RDMA实现。
50%UnverifiedInfiniBand是一种高吞吐、低延迟的网络互连标准，广泛应用于超算和AI训练集群。
50%UnverifiedAMD's ROCm ecosystem achieves RDMA capability by supporting RDMA over Converged Ethernet (RoCE), competing with InfiniBand-based solutions.
50%UnverifiedGPUs transmit gradient data via InfiniBand or RoCE protocols, with bandwidth reaching 400 Gbps per card or higher.
50%UnverifiedTensor parallelism is typically used only between NVLink-connected GPUs within the same node; cross-node deployment severely slows training throughput due to insufficient InfiniBand bandwidth
50%UnverifiedNVIDIA's InfiniBand network and RoCEv2-based solutions have become the mainstream choices for large-scale AI training clusters.
50%UnverifiedRoCE is an implementation of RDMA over standard Ethernet, with lower deployment costs compared to traditional RDMA deployments that require dedicated InfiniBand networks.
50%UnverifiedWhen multiple GPUs are interconnected via NVLink or InfiniBand, the interconnect bandwidth often becomes the hard ceiling for end-to-end inference throughput rather than GPU compute itself.
50%UnverifiedIn traditional networking solutions, network jitter and protocol overhead typically result in actual bandwidth utilization of only 60%–70%
50%UnverifiedTraditional GPU servers need to transfer data across hundreds to thousands of chips via high-speed interconnect protocols such as NVLink and InfiniBand, which consumes significant bandwidth and introduces latency and additional power consumption.
50%UnverifiedThe core competitiveness of neoclouds lies in high-density InfiniBand network-interconnected GPU clusters, more flexible compute rental periods, and lower barriers to entry.
50%UnverifiedExpert Parallelism places high demands on interconnect bandwidth (such as NVLink and InfiniBand).
50%