Concept快速启动 / ipc_cache
Fast Start
A fast-start feature in vLLM that significantly reduces cold-start time during service restarts by storing quantized weights in a per-GPU resident daemon process and mapping them directly via CUDA IPC.
Timeline (last 90 days)
Sep 24
The Fast Start feature in v0.30.0 saves quantized, tensor-parallel-sharded weights in a per-GPU resident daemon process and maps them directly via CUDA IPC on restart, enabled with --load-format ipc_cache.
Unverified50%
Sep 24
Fast Start can compress cold start time from minutes to seconds.
Unverified50%