Unverified50% confidenceFactExact time
The H100's peak arithmetic intensity is approximately 295 FLOPs/Byte; when the batch size exceeds this value it is compute-bound, and when it is below this value it is memory-bound.
1
Sources
50%
Confidence
Long-term
Relevance
9/23/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe arithmetic intensity of the decoding stage in LLM autoregressive generation is extremely low: for a 7B-parameter FP16 model, one forward pass requires loading approximately 14 GB of weights but executes only about 14 GFLOPs of computation, resulting in an arithmetic intensity of approximately 1 FLOP/Byte, making it a bandwidth-bound workload.65% similarVerifiedThe NVIDIA H20's peak BF16 compute is approximately 296 TFLOPS, significantly reduced compared to the H100's 989 TFLOPS, but it retains 141 GB HBM3 memory and 4.0 TB/s memory bandwidth.64% similarUnverifiedThe H20 has 96GB HBM3 memory, and the L20's FP8 computing performance is approximately half that of the H20.63% similarUnverifiedLlama 2 13B on an H100 at batch size 1 has a latency of approximately 8 milliseconds per token and a throughput of approximately 124 tokens/second, while at batch size 256 it requires approximately 240 GB of memory, which does not fit on a single H100.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/942433API
curl https://kongchang.com/api/v1/knowledge/claims/942433MCP
get_claim(id=942433)