Unverified50% confidenceFactExact time
KLD is sensitive to the distribution tails: as long as the quantized model assigns extremely low probability to low-probability tokens, KLD will be significantly inflated, even if those tokens have little impact on generation quality.
1
Sources
50%
Confidence
Long-term
Relevance
10/24/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe relationship between KLD and downstream benchmark performance is not linear: when KLD is below a certain magnitude, downstream metrics are almost unaffected; only when KLD rises significantly does task performance begin to decline markedly.71% similarUnverified该测试采用KL散度（KLD）作为客观量化指标，衡量量化后输出分布与BF16参考分布的偏离程度，KLD越低说明量化造成的信息损失越小70% similarUnverifiedUnbalanced optimal transport allows marginal constraints to be softly satisfied by introducing penalty terms such as KL divergence into the objective function, giving lower-confidence matches smaller weights.67% similarUnverifiedBrownian diffusion models may systematically underestimate the probability of extreme events in heavy-tailed data scenarios, whereas stable models can correctly extrapolate to the tail region when α=ν.67% similarUnverified对MoE模型的量化需要格外谨慎，因稀疏激活使专家参数数值分布集中，低比特量化易造成路由偏差导致能力退化66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1094694API
curl https://kongchang.com/api/v1/knowledge/claims/1094694MCP
get_claim(id=1094694)