ConceptPipeline Parallelism / 流水线并行
Pipeline Parallelism
A distributed model inference and training parallelism strategy that partitions a model by layer across different devices, with each device responsible for the computation of different layers.
A distributed model inference and training parallelism strategy that partitions a model by layer across different devices, with each device responsible for the computation of different layers.