KAI Scheduler
A GPU scheduler for AI workloads that supports gang scheduling, queue priorities, fair-share and preemption policies, and natively integrates GPU fractionalization.
Timeline (last 90 days)
The open-source release contains four modules: KAI Scheduler, the GPU fractioning capability in CUDA 13.4, quota management, and a local Kind development environment.
KAI Scheduler originated from an internal scheduler project at Run:ai and uses gang scheduling, meaning all workers of a distributed training job are either scheduled at the same time or all wait.
KAI Scheduler provides queue and priority hierarchies, fair-share allocation, and preemption policies, where high-priority tasks can evict low-priority tasks and reclaim their GPUs.
Open-source solutions such as Volcano and Gang Scheduling already have precedents for concepts like queues, fair share, and preemption; KAI Scheduler's differentiator is its native integration with GPU fractionalization and the quota system.
All Facts (4)
The open-source release contains four modules: KAI Scheduler, the GPU fractioning capability in CUDA 13.4, quota management, and a local Kind development environment.
50%UnverifiedKAI Scheduler originated from an internal scheduler project at Run:ai and uses gang scheduling, meaning all workers of a distributed training job are either scheduled at the same time or all wait.
50%UnverifiedKAI Scheduler provides queue and priority hierarchies, fair-share allocation, and preemption policies, where high-priority tasks can evict low-priority tasks and reclaim their GPUs.
50%UnverifiedOpen-source solutions such as Volcano and Gang Scheduling already have precedents for concepts like queues, fair share, and preemption; KAI Scheduler's differentiator is its native integration with GPU fractionalization and the quota system.
50%