Unverified50% confidenceBenchmarkExact time
在有缺陷的蒸馏目标下，KD loss为2.729，普通交叉熵基线仅为2.081，即KD表现不如CE基线
1
Sources
50%
Confidence
Long-term
Relevance
7/14/2026
First Seen
Sources
跨分词器知识蒸馏：85%信息丢失的根因与链式法则解法
Related Claims
VerifiedCross-entropy loss originates from KL divergence in information theory and measures the degree of difference between the true probability distribution and the predicted probability distribution.61% similarUnverifiedThe choice of K value in Top-K routing affects the trade-off between computational efficiency and expressive capacity — a smaller K leads to faster inference but may sacrifice expressiveness, and in practice K is typically set to 1 or 2.60% similarUnverified该测试采用KL散度（KLD）作为客观量化指标，衡量量化后输出分布与BF16参考分布的偏离程度，KLD越低说明量化造成的信息损失越小60% similarUnverifiedVAE的损失函数由重建损失（常用MSE或BCE）和KL散度两部分构成，KL散度约束潜在分布接近标准正态分布起正则化作用60% similarUnverified如果交叉熵稳步下降而Dice波动剧烈，问题几乎可以确定出在Dice损失的配置上58% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/508982API
curl https://kongchang.com/api/v1/knowledge/claims/508982MCP
get_claim(id=508982)