Unverified50% confidenceFactExact time
In knowledge distillation, when the student logits exactly equal the teacher logits, the standard softmax implementation produces non-zero spurious gradients.
1
Sources
50%
Confidence
Long-term
Relevance
10/23/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSoftmax函数具有平移不变性，任意放大所有logit的尺度不改变预测类别，却会将输出概率压向极端值，是其天然的过度自信倾向68% similarUnverifiedAfter softmax normalization, the logits output by the cross-encoder have an exponential amplifying effect on citation probability, so tiny differences in vector similarity are insufficient to decide the outcome.67% similarUnverified该神经网络能力上限被Minimax数据锁定，最多只能逼近教师算法水平而无法超越66% similarVerifiedIf samples are too easy the model has nothing to learn; if they are too difficult, noisy learning signals may interfere with training; the most effective learning occurs in the region slightly above current capability but still within reach.63% similarUnverifiedLLM进行数学计算时依赖从训练数据学到的模式匹配，而非执行确定性算术逻辑，面对罕见组合（如大数乘法）准确率会急剧下降63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1088266API
curl https://kongchang.com/api/v1/knowledge/claims/1088266MCP
get_claim(id=1088266)