Verified65% confidenceFactExact time
连续批处理（Continuous Batching）采用迭代级别的调度粒度，每完成一个 token 生成便可移出已完成请求并插入新请求
3
Sources
65%
Confidence
Long-term
Relevance
7/17/2026
First Seen
Sources
Related Claims
Unverified大模型服务商通常采用批处理技术提高吞吐量，连续批处理允许新请求动态插入正在处理的批次，而静态批处理需等待完整批次处理完毕75% similarVerifiedvLLM implements a Continuous Batching mechanism that allows new requests to be inserted immediately after some requests in a batch are completed, without waiting for the entire batch to finish.74% similarUnverifiedContinuous Batching allows new requests to be inserted into an existing batch in-flight and is implemented by inference engines such as vLLM and TensorRT-LLM.74% similarVerifiedKV Cache通过将已计算的键值对持久化存储，使相同Prompt前缀的后续请求可直接复用，跳过重复的矩阵运算70% similarUnverifiedContinuous batching allows dynamic adjustment of the batch at each decoding step, immediately inserting a newly waiting request once a request completes; Orca (a scheduling system proposed in 2022) was the first to implement this idea.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/544088API
curl https://kongchang.com/api/v1/knowledge/claims/544088MCP
get_claim(id=544088)