Unverified50% confidenceFactExact time
Existing research has found that a linear attention layer appropriately pretrained on linear regression tasks effectively implements the functionality of one-step gradient descent.
1
Sources
50%
Confidence
Long-term
Relevance
10/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedALiBi在注意力分数中加入与距离成线性比例的负偏置项，实现近程优先的注意力模式，且外推长度几乎不受训练长度限制73% similarUnverified经验回放将交互经验存入固定大小缓冲区并随机采样小批量训练，打破连续经验的时间相关性，使神经网络训练更稳定73% similarUnverifiedWhen the distribution of online data deviates from the training data, model performance degrades (data drift and model decay), requiring retraining to be triggered.71% similarUnverifiedLinear probes perform well on the training distribution, but their performance often drops sharply when facing out-of-distribution (OOD) samples.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/990808API
curl https://kongchang.com/api/v1/knowledge/claims/990808MCP
get_claim(id=990808)