Unverified50% confidenceFactExact time
The decode phase at batch size 1 is compute-bound, so placement strategies have virtually no effect on throughput and only alter PCIe migration traffic and time-to-first-token (TTFT).
1
Sources
50%
Confidence
Long-term
Relevance
9/16/2026
First Seen
Sources
Related Entities
Related Claims
Unverified传统混合部署中Prefill阶段的大批量计算会阻塞Decode阶段的token生成，导致TTFT波动剧烈68% similarUnverifiedModern large model inference typically uses batching technology to improve GPU utilization, and backlogs in the batching queue during peak periods cause the time to first token (TTFT) to become longer.67% similarVerifiedThe speed bottleneck of model inference on mobile devices is often not computing power but memory bandwidth.66% similarUnverified对流式响应场景，应重点关注首Token延迟（TTFT）而非整体完成时间65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/924729API
curl https://kongchang.com/api/v1/knowledge/claims/924729MCP
get_claim(id=924729)