Serverless GPU
A cloud computing model that allocates GPU resources on demand without requiring reserved instances. Resources are automatically activated upon request and billed by actual usage time, making the model suitable for AI inference workloads with variable or unpredictable traffic patterns.
Timeline (last 90 days)
Databricks' solution deploys open-weight decision models onto Serverless GPUs through AI Runtime and Model Serving.
Serverless GPUs can be spun up on demand and released when done, so teams do not need to pay for idle GPU resources or worry about underlying scheduling.
Cold Start refers to the latency period from when an instance is spun up, the container is loaded, and the model weights are loaded, until it is able to respond to inference requests.
All Facts (3)
Databricks' solution deploys open-weight decision models onto Serverless GPUs through AI Runtime and Model Serving.
50%UnverifiedServerless GPUs can be spun up on demand and released when done, so teams do not need to pay for idle GPU resources or worry about underlying scheduling.
50%UnverifiedCold Start refers to the latency period from when an instance is spun up, the container is loaded, and the model weights are loaded, until it is able to respond to inference requests.
50%