Agentic Serving

An inference serving mode for AI agent workloads. Unlike single-turn question-answering inference, an agent may issue dozens of model calls within one task, forming a think-act-observe loop, which requires the engine to support prefix cache sharing, continuous batching and priority preemption scheduling.