Unverified50% confidenceFactExact time
在计算交叉熵损失时，softmax对正确标签位置i的梯度为p_i-1，对其他位置j的梯度为p_j，使lm_head输出层梯度密集
1
Sources
50%
Confidence
Long-term
Relevance
8/24/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedCross-entropy loss often converges faster than MSE in classification tasks because, when paired with softmax output, gradients do not vanish in the saturation region.71% similarUnverifiedWhen a model encounters out-of-distribution inputs, prediction entropy increases, the softmax output tends toward a uniform distribution, and the range of candidate tokens permitted by nucleus sampling expands, giving long-tail anomalous tokens a chance of being sampled.62% similarVerifiedHNSW reduces search complexity to approximately O(log n) by constructing a multi-layer graph index, while IVF partitions the vector space via K-means clustering to achieve orders-of-magnitude speed improvements at the cost of approximately 5–10% recall.61% similarVerifiedCross-Entropy Loss is commonly used for classification tasks, and Mean Squared Error (MSE) is commonly used for regression tasks.61% similarUnverifiedForge性能比预期慢，主要瓶颈集中在交叉熵损失函数及其梯度计算和softmax上，且尚未实现KV-cache61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/793873API
curl https://kongchang.com/api/v1/knowledge/claims/793873MCP
get_claim(id=793873)