Pipeline parallelism splits the model into multiple stages by layer, places different stages on different devices, and executes them in a micro-batch pipeline, which relieves memory pressure but suffers from pipeline bubbles.