Unverified60% confidenceFactExact time
The softmax operation is prone to numerical overflow or underflow due to insufficient precision when handling extreme values, which is the core technical reason why early FP8 quantization performed poorly.
2
Sources
60%
Confidence
Long-term
Relevance
7/11/2026
First Seen
Sources
Krea2 Tested: Does the FP8 vs BF16 Quality Gap Still Exist?
redditr/StableDiffusion7/9/2026
Related Claims
Unverified计算 softmax 函数时指数运算可能导致浮点数溢出，标准做法是先减去输入向量的最大值再取指数79% similarUnverifiedSoftmax involves exponential operations, row-wise normalization, and global dependencies, making it difficult to parallelize and quantize, and it is especially resource-intensive in long-sequence scenarios78% similarUnverifiedSoftmax输出的概率值并不等同于真实的贝叶斯后验概率，而是被过度优化以拉开类别间距，导致预测概率系统性偏高77% similarUnverifiedSoftmax函数具有平移不变性，任意放大所有logit的尺度不改变预测类别，却会将输出概率压向极端值，是其天然的过度自信倾向72% similarUnverifiedLow-bit representation significantly amplifies the impact of outliers in weight distributions; SmoothQuant addresses this by migrating quantization difficulty, while SpQR addresses it by separately retaining high precision for outliers.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/475413API
curl https://kongchang.com/api/v1/knowledge/claims/475413MCP
get_claim(id=475413)