Unverified50% confidenceFactExact time
During the LLM sequential decoding stage, a pipeline-parallel cluster exchanges activation tensors rather than model weights; taking hidden dimension d=3584 as an example, at FP32 precision this is about 14.3KB per token, and after symmetric INT8 dynamic quantization it is only 5.1KB per hop on the network.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe arithmetic intensity of the decoding stage in LLM autoregressive generation is extremely low: for a 7B-parameter FP16 model, one forward pass requires loading approximately 14 GB of weights but executes only about 14 GFLOPs of computation, resulting in an arithmetic intensity of approximately 1 FLOP/Byte, making it a bandwidth-bound workload.76% similarUnverifiedOn standard gigabit Ethernet, one pipeline-parallel hop is about 0.45ms, and a full 3 hops is about 7.88ms, accounting for less than 0.2% of total token execution time, so gigabit Ethernet is completely sufficient for distributed inference, and the bottleneck is not the network.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1010447API
curl https://kongchang.com/api/v1/knowledge/claims/1010447MCP
get_claim(id=1010447)