Unverified50% confidenceFactExact time
First-order Taylor importance estimation approximates the impact on loss of removing a given attention dimension by the product of "gradient x activation value"; the smaller the absolute value of the product, the weaker that dimension's contribution.
1
Sources
50%
Confidence
Long-term
Relevance
11/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe endpoint metric used in the evaluation is the difference of 'model minus uniform control group', where a positive value represents performing worse than the control.72% similarUnverifiedIQR is insensitive to extreme values and is particularly suitable for describing target area distributions that may exhibit a long tail or even zero values after cropping; when augmentation causes the targets in some samples to nearly disappear, the standard deviation is significantly inflated by outliers, whereas IQR still robustly reflects the dispersion of the main body of samples.70% similarUnverifiedUnder fixed compute, single-head attention scores 0.9 points lower than the optimal setting, and quality actually decreases when there are too many heads.69% similarUnverifiedPercentage tolerance makes inputs of different magnitudes comparable and decouples thresholds from numerical scale, but the cost is that small absolute fluctuations in small numbers are amplified into high percentages, while large absolute changes in large numbers may only produce moderate percentages.69% similarUnverifiedModule residual anchoring lowered the hit rate under natural questioning from 28.7% to 22.0% while also reducing the measured behavioral drift, a trade-off of exchanging recall for preservation of non-target behavior.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1130517API
curl https://kongchang.com/api/v1/knowledge/claims/1130517MCP
get_claim(id=1130517)