Unverified50% confidenceTradeoffExact time
Tensor parallelism is typically used only between NVLink-connected GPUs within the same node; cross-node deployment severely slows training throughput due to insufficient InfiniBand bandwidth
1
Sources
50%
Confidence
Long-term
Relevance
10/6/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedNVLink's bandwidth is far lower than the intra-card bandwidth of a single GPU, and cross-card communication creates a bottleneck.73% similarUnverifiedTensor parallelism requires extremely high inter-node bandwidth (typically relying on NVLink or InfiniBand) and is unsuitable for high-latency public network environments; pipeline parallelism vertically partitions by layer, has lower peak bandwidth requirements, and is better suited for heterogeneous network environments.72% similarUnverifiedNVLink speed degradation or fallback to PCIe mode can reduce intra-node communication speed by several times, yet OS-level health checks will still mark the link as normal.66% similarUnverifiedNVLink Switch的拓扑并非完全扁平，经过不同层级Switch的通信路径在延迟和有效带宽上存在差异65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/982623API
curl https://kongchang.com/api/v1/knowledge/claims/982623MCP
get_claim(id=982623)