NVLink
NVLink is a high-speed chip-to-chip interconnect technology developed by NVIDIA, enabling high-bandwidth, low-latency direct communication between multiple GPUs or between GPUs and CPUs. Compared to conventional PCIe buses, NVLink provides higher data transfer bandwidth and lower latency, supporting efficient data sharing and communication in multi-GPU configurations. It is widely used in high-performance computing, deep learning training, and data center environments.
Core Facts
NVLink 4.0 offers a bidirectional bandwidth of up to 900 GB/s, while cross-node InfiniBand HDR offers only approximately 50 GB/s, a gap of nearly 20 times.
在H100 NVLink架构中，单台服务器内8块GPU可通过NVSwitch全互联，总带宽达900GB/s，相比PCIe 5.0的64GB/s有数量级提升
NVIDIA's latest NVLink 5.0 can provide 1.8 TB/s bidirectional bandwidth per GPU.
Timeline (last 90 days)
Tensor parallelism splits the weight matrix of the same layer and places it across multiple machines, requiring one All-Reduce communication per layer computed, is extremely sensitive to bandwidth and latency, and is usually only practical under high-speed interconnects such as NVLink.
RTX 3090 是最后一张带物理 NVLink 接口的消费卡；两张 3090 加桥接后可获得 112.5 GB/s 双向带宽。
Tensor parallelism is typically used only between NVLink-connected GPUs within the same node; cross-node deployment severely slows training throughput due to insufficient InfiniBand bandwidth
专业AI服务器通过NVLink实现GPU直连，带宽可达数百GB/s
AMD's hardware interconnect technology is xGMI/Infinity Fabric, while NVIDIA's is NVLink.
When multiple GPUs are interconnected via NVLink or InfiniBand, the interconnect bandwidth often becomes the hard ceiling for end-to-end inference throughput rather than GPU compute itself.
The NVIDIA version of DeepEP uses NCCL, with NVLink within a node, RDMA across nodes, and kernels written in CUDA.
NVIDIA has designed NVLink high-speed interconnects and the NCCL communication library specifically for multi-card training, forming a complete hardware-software solution.
UB (Unified Bus) is a high-speed inter-chip interconnect technology unique to the Ascend platform, serving a similar role to NVIDIA's NVLink but implemented differently
Traditional GPU servers need to transfer data across hundreds to thousands of chips via high-speed interconnect protocols such as NVLink and InfiniBand, which consumes significant bandwidth and introduces latency and additional power consumption.
24 more timeline events
All Facts (20)
NVLink 4.0 offers a bidirectional bandwidth of up to 900 GB/s, while cross-node InfiniBand HDR offers only approximately 50 GB/s, a gap of nearly 20 times.
80%Verified在H100 NVLink架构中，单台服务器内8块GPU可通过NVSwitch全互联，总带宽达900GB/s，相比PCIe 5.0的64GB/s有数量级提升
75%VerifiedNVIDIA's latest NVLink 5.0 can provide 1.8 TB/s bidirectional bandwidth per GPU.
75%VerifiedNVLink 4.0 (used in the H100) can achieve a unidirectional bandwidth of 900 GB/s.
70%VerifiedIn benchmark testing, Qwen3 27B achieved a generation speed of approximately 40-something tokens per second on 4 modified 2080 Ti cards without NVLink enabled.
65%VerifiedHigh-end training clusters dominated by NVIDIA rely on InfiniBand high-speed networks (with bandwidth up to 800Gb/s) and NVLink/NVSwitch to achieve direct GPU interconnection
65%VerifiedNVLink 4.0的双向带宽高达900GB/s，而传统PCIe总线带宽约为64GB/s
65%UnverifiedNVLink技术自2016年Pascal架构起推出，旨在突破PCIe总线的带宽限制
95%UnverifiedHopper架构的第四代NVLink提供900GB/s的带宽
90%UnverifiedNCCL通信库已经针对NVLink拓扑做了深度优化
90%Unverified传统DGX集群中，8块GPU组成一个节点，节点内走NVLink，节点间走InfiniBand
90%Unverified大语言模型训练背后需要精密的数据中心电源分配单元、液冷或风冷散热系统、高速互联网络（NVLink、InfiniBand）以及UPS不间断电源保障
90%Unverified块调度建议将张量并行（TP）限制在NVLink带宽最高的GPU子集内（同一个最小块），数据并行（DP）跨块分布，流水线并行（PP）沿拓扑层次展开
85%UnverifiedGB200 NVL72的72块GPU可以按照NVLink拓扑结构划分为不同粒度的块，包括9个8-GPU块、4个18-GPU块、2个36-GPU块或1个72-GPU块
85%Unverified在大规模训练集群中，节点内通信（NVLink）与节点间通信（InfiniBand或RoCE）的性能差异可达一个数量级
80%UnverifiedThe core goal of NVLink is to break through the bandwidth limitations of the traditional PCIe bus and allow multiple GPUs to communicate directly at extremely high speeds.
70%UnverifiedGB200 NVL72将NVLink互联从单节点内部扩展到了整个机架级别，机架内任意两块GPU之间都能通过NVLink直接通信
70%UnverifiedThe fifth generation of NVLink (featured in the Blackwell architecture) offers a single-link bidirectional bandwidth of up to 1.8 TB/s.
70%UnverifiedThe Blackwell architecture increases NVLink bandwidth to 1.8 TB/s per GPU
70%UnverifiedTraining requires GPU clusters with high-bandwidth interconnects (such as A100/H100 connected via NVLink), while inference places greater emphasis on single-card memory bandwidth, batch processing throughput, and low latency.
60%