Unverified50% confidenceFactExact time
蒸馏的核心洞察在于神经网络的软标签（类别概率分布）包含比one-hot硬标签更丰富的结构信息
1
Sources
50%
Confidence
Long-term
Relevance
7/22/2026
First Seen
Sources
Related Claims
Unverified叠加现象指神经网络倾向于用远少于特征数量的神经元同时编码大量语义特征，与传统一个神经元一个特征的直觉相悖68% similarUnverifiedSoftmax函数将神经网络logits压缩为概率分布，但其输出值在大规模模型中系统性偏离真实贝叶斯后验概率66% similarUnverifiedFID (Fréchet Inception Distance) evaluates generation quality by comparing the statistical distribution differences between real and generated data in the feature space of a deep neural network, with lower values indicating closer resemblance to real data.64% similarUnverifiedNeural networks have an overconfidence defect: even when inputs fall outside the training distribution, the Softmax layer may still output inflated confidence scores.63% similarUnverified在神经网络推理中Softmax、LayerNorm等对数值范围敏感的操作极易因FP16有限的指数位引发上溢或下溢，误差在Transformer多层堆叠中累积放大62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/592756API
curl https://kongchang.com/api/v1/knowledge/claims/592756MCP
get_claim(id=592756)