Expert Adapters
Expert adapters are LoRA adapters attached individually to each expert in an MoE model, which can improve inference efficiency.
Timeline (last 90 days)
vLLM improves decoding throughput without changing the model structure by organizing each expert's adapters into batches and scheduling them fused with expert computation to reduce extra overhead.
The 62% decoding throughput improvement is highly dependent on the specific configuration; the number of experts, routing distribution, and concurrency level all affect adapter scheduling efficiency.
All Facts (2)
vLLM improves decoding throughput without changing the model structure by organizing each expert's adapters into batches and scheduling them fused with expert computation to reduce extra overhead.
50%UnverifiedThe 62% decoding throughput improvement is highly dependent on the specific configuration; the number of experts, routing distribution, and concurrency level all affect adapter scheduling efficiency.
50%