Serverless Inference
An AI model inference deployment mode in which model instances are started on demand and destroyed after use, with the platform responsible for elastic scaling. Users do not need to manage underlying GPU resources, making it suitable for intermittent workloads.
Timeline (last 90 days)
Serverless Inference is suitable for scenarios with sparse or bursty traffic and is billed by number of invocations.
In serverless inference mode, model instances are started on demand and destroyed after use, with the platform handling elastic scaling; in dedicated inference mode, users reserve fixed GPU instances for a specific model that run continuously
All Facts (2)
Serverless Inference is suitable for scenarios with sparse or bursty traffic and is billed by number of invocations.
50%UnverifiedIn serverless inference mode, model instances are started on demand and destroyed after use, with the platform handling elastic scaling; in dedicated inference mode, users reserve fixed GPU instances for a specific model that run continuously
50%