Unverified50% confidenceFactExact time
Attention mass remaining on the evidence explains at most 58% of the variance in retrieval loss across 126,473 trials, while log length explains at most 8%.
1
Sources
50%
Confidence
Long-term
Relevance
11/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedUnder fixed compute, single-head attention scores 0.9 points lower than the optimal setting, and quality actually decreases when there are too many heads.72% similarUnverifiedAccording to the ForgetEval research, when leading memory systems were scored on test cases written by people other than the authors themselves, internal test scores of 63%-68% plummeted to 28%-33%.70% similarUnverifiedFirst-order Taylor importance estimation approximates the impact on loss of removing a given attention dimension by the product of "gradient x activation value"; the smaller the absolute value of the product, the weaker that dimension's contribution.70% similarUnverifiedCausal probes achieved a 17%–118% improvement in steering capability across different models, with only a 3% decrease in concept detection capability.69% similarUnverifiedIn the composite score for memory retrieval, similarity has the largest weight, recency accounts for a small proportion, and the result is multiplied by an importance factor with a range of approximately 0.8 to 1.269% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1134063API
curl https://kongchang.com/api/v1/knowledge/claims/1134063MCP
get_claim(id=1134063)