Unverified50% confidenceFactExact time
An information-theoretic contrastive objective usually refers to a loss function that measures distributional differences such as KL divergence or cross-entropy, and NatScale uses the calibrated weights as the error penalty coefficients in these objectives.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedCross-entropy loss originates from KL divergence in information theory and measures the degree of difference between the true probability distribution and the predicted probability distribution.76% similarVerifiedThe core loss functions of contrastive learning (such as InfoNCE and NT-Xent) essentially pull the representations of similar samples closer together while pushing the representations of dissimilar samples further apart.72% similarUnverifiedNatScale combines the calibrated weights and error estimates with an existing prior that covers three dimensions: weight spectrum, variance, and precision decay.70% similarUnverified监督学习本质上是函数逼近问题，通过最小化损失函数实现，常见损失函数包括回归任务的均方误差（MSE）和分类任务的交叉熵（Cross Entropy）68% similarUnverified该测试采用KL散度（KLD）作为客观量化指标，衡量量化后输出分布与BF16参考分布的偏离程度，KLD越低说明量化造成的信息损失越小68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1032958API
curl https://kongchang.com/api/v1/knowledge/claims/1032958MCP
get_claim(id=1032958)