Unverified50% confidenceFactExact time
Floating-point addition and multiplication are not associative under the IEEE 754 standard, and the different internal summation orders of GEMM and GEMV produce extremely small differences in activation values for the same input.
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
Unverified浮点运算不满足结合律，不同线程束的执行顺序差异会导致数值结果出现细微偏差64% similarUnverifiedIn large-scale GPU matrix operations, the associative law of floating-point addition does not strictly hold, and different parallel reduction orders, batch sizes, and memory scheduling can cause minor discrepancies in logits.60% similarUnverifiedFP8 has a narrower numerical range and lower precision, and correct usage requires fine-grained scaling factor management and numerical stability guarantees at the GEMM kernel level.59% similarVerifiedIn GPU parallel reduction, because floating-point operations do not satisfy the associative law, differences in the execution order of different warps cause deviations in numerical results, which is the root cause of non-determinism.58% similarUnverifiedGroup GEMM is the bottleneck of MoE computation, as the differing numbers of tokens received by different experts require simultaneously executing a batch of small matrix multiplications with varying shapes.57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1000272API
curl https://kongchang.com/api/v1/knowledge/claims/1000272MCP
get_claim(id=1000272)