Unverified50% confidenceFactExact time
When running LLMs across multiple GPUs in parallel, tensor parallelism is limited by PCIe bandwidth and latency due to frequent synchronization of activations between cards; pipeline parallelism has pipeline bubbles, and utilization depends on the number of micro-batches.
1
Sources
50%
Confidence
Long-term
Relevance
11/1/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedA dense 35B model requires more than about 7 billion multiply-add operations per token, and latency on a single card may be as high as hundreds of milliseconds or even seconds.64% similarUnverifiedAn LLM gateway adds one network hop of latency, but relative to generation time (hundreds to thousands of milliseconds) this is usually small.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1122975API
curl https://kongchang.com/api/v1/knowledge/claims/1122975MCP
get_claim(id=1122975)