Unverified50% confidenceFactExact time
Entropy is sensitive to temperature scaling and logit scale, and a model may lower entropy by amplifying logit magnitudes rather than improving feature representations.
1
Sources
50%
Confidence
Long-term
Relevance
11/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedUnder the constrained optimization framework, the inverse temperature parameter of temperature sampling corresponds exactly to the Lagrange multiplier of the cross-entropy budget constraint.69% similarUnverifiedIn the context of attention, the temperature parameter controls the sharpness of the softmax: low temperature makes attention concentrate on a few keys (low entropy, possibly overfitting), while high temperature makes the weights more uniform (high entropy, possibly under-focusing).65% similarUnverified温度是0到2之间的超参数，其底层机制是在Softmax函数前对logits进行缩放，温度趋近0时概率分布趋向one-hot，温度升高时分布趋于平坦64% similarVerifiedTemperature scaling softens the prediction distribution by dividing by a learned temperature parameter T before the softmax, but its applicability to LLMs is limited by the unstructured nature of the output.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1129521API
curl https://kongchang.com/api/v1/knowledge/claims/1129521MCP
get_claim(id=1129521)