Unverified50% confidenceFactExact time
Tensor Parallelism (TP) splits individual Transformer layer weight matrices across different GPUs, requiring All-Reduce communication per layer forward pass; Pipeline Parallelism (PP) splits the model by layers with lower communication volume but has bubbles that reduce utilization
1
Sources
50%
Confidence
Long-term
Relevance
9/10/2026
First Seen
Sources
Related Entities
Related Claims
Unverified流水线并行按模型的不同层将计算分配到不同设备，通信量仅为相邻层的激活张量，但存在流水线气泡问题71% similarUnverified张量并行将单层权重切分到多卡，延迟低但通信开销大；流水线并行按层切分，吞吐高但首token延迟长67% similarUnverifiedModel parallelism consists of two approaches: tensor parallelism, which splits tensors within a single layer across multiple cards, and pipeline parallelism, which assigns different layers to different devices.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/887071API
curl https://kongchang.com/api/v1/knowledge/claims/887071MCP
get_claim(id=887071)