Unverified50% confidenceBenchmarkExact time
After replacing the intermediate attention modules with OAL and jointly fine-tuning via LoRA, the WikiText-103 perplexity dropped from 31.71 to 27.50 compared to the global quadratic kernel.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAt moderate noise levels and the middle layers of the model, 2 learnable null slots absorbed approximately 90% of the cross-attention mass, while the EOS token's share dropped sharply to approximately 4%.59% similarUnverifiedAfter rebuilding the configuration under the same public identifier on the same day, the model-minus-uniform endpoint value decreased by 0.0429 (H-minus-R comparison 95% CI: [+0.0182, +0.0667]), and with six configuration components changing simultaneously it was impossible to isolate which component caused the difference.56% similarUnverifiedThe LoRA baseline used for comparison in the paper uses rank 19 with 6.6 million parameters, achieving a correction rate of 83.3%, but exhibited capability loss of 30% to 75%.55% similarUnverifiedAfter applying DA, Gemma-4-31B reduced attention tokens by 52.0%, with an accuracy drop of only 1.27 percentage points.55% similarUnverified在WikiText-103数据集上，约10M参数下CWAA V6验证困惑度为139.09，标准Transformer为149.19，CWAA约有7%相对改善54% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1002308API
curl https://kongchang.com/api/v1/knowledge/claims/1002308MCP
get_claim(id=1002308)