Unverified50% confidenceFactExact time
交叉熵恢复率通过将SAE重建激活替代原始激活输入到模型后续层比较交叉熵损失变化来计算，衡量SAE是否保留对下游计算重要的信息
1
Sources
50%
Confidence
Long-term
Relevance
8/13/2026
First Seen
Sources
Related Claims
UnverifiedThe semantic entropy method is technically implemented by obtaining candidate answers through multiple samplings, using an NLI model to determine semantic equivalence for clustering, and then computing entropy over the semantic categories.66% similarUnverifiedThe binary cross-entropy loss formula is -[y·log(p) + (1-y)·log(1-p)], where y is the true label and p is the model's predicted probability for the positive class.65% similarUnverifiedThe degree of model memorization is highly correlated with the number of data repetitions; when the number of parameters greatly exceeds the information entropy of the training data, the model tends to memorize rather than generalize.64% similarUnverifiedMemGAS uses an Entropy Router to adaptively select the optimal memory granularity based on the entropy of the query's similarity distribution across different granularities.63% similarUnverifiedThe parameters of logistic regression are estimated by maximizing the log-likelihood function (equivalent to minimizing the cross-entropy loss).62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/742346API
curl https://kongchang.com/api/v1/knowledge/claims/742346MCP
get_claim(id=742346)