FluidPD
An LLM inference optimization paper published on arXiv that proposes an in-place elastic scheduling solution for resource imbalance in Prefill-Decode disaggregated architectures. It introduces two mechanisms, FluidToken and FluidRole, to achieve SLO-aware dynamic resource reallocation between prefill and decode workers.
Timeline (last 90 days)
FluidPD is a paper on arXiv that proposes an in-place elastic scheduling solution targeting resource imbalance in P/D disaggregation architectures for LLM inference.
The paper identifies three shortcomings of traditional auto-scaling: slow reaction time, the need to reserve idle GPUs, and the inability to directly address short-timescale phase imbalances.
FluidPD introduces two complementary mechanisms, FluidToken and FluidRole, to address resource imbalances at different timescales.
The scheduling of FluidPD's two mechanisms is guided by a lightweight pressure index that intervenes proactively before SLOs are violated.
The paper evaluates on production-level Azure trace workloads and shows that FluidPD improves overall SLO attainment by up to 94.6 percentage points compared to statically configured SGLang.
FluidPD can significantly improve service quality without provisioning additional workers.
FluidPD is a newly published arXiv preprint, and its conclusions are pending broader peer review and real-world deployment validation.
All Facts (7)
FluidPD is a paper on arXiv that proposes an in-place elastic scheduling solution targeting resource imbalance in P/D disaggregation architectures for LLM inference.
50%UnverifiedThe paper identifies three shortcomings of traditional auto-scaling: slow reaction time, the need to reserve idle GPUs, and the inability to directly address short-timescale phase imbalances.
50%UnverifiedFluidPD introduces two complementary mechanisms, FluidToken and FluidRole, to address resource imbalances at different timescales.
50%UnverifiedThe scheduling of FluidPD's two mechanisms is guided by a lightweight pressure index that intervenes proactively before SLOs are violated.
50%UnverifiedThe paper evaluates on production-level Azure trace workloads and shows that FluidPD improves overall SLO attainment by up to 94.6 percentage points compared to statically configured SGLang.
50%UnverifiedFluidPD can significantly improve service quality without provisioning additional workers.
50%UnverifiedFluidPD is a newly published arXiv preprint, and its conclusions are pending broader peer review and real-world deployment validation.
50%