Product
DistServe
A disaggregated deployment system for LLM inference that places the prefill phase and the decode phase on separate GPU resource pools, improving inference efficiency and service quality by allowing each phase to be optimized independently.
Timeline (last 90 days)
Sep 16
Systems such as DistServe, Splitwise, and Mooncake have already implemented disaggregated deployment of prefill and decode.
Unverified50%
Sep 10
Splitwise, DistServe, and Mooncake are all representative implementations of prefill-decode (PD) disaggregation.
Unverified50%