Unverified50% confidenceFactExact time
Performer and Linear Transformer are early representative works on linearized attention, and generally suffer from rising perplexity and degraded long-range dependency modeling.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedPrompt drift refers to the phenomenon in which model behavior gradually deviates from the original intent of instructions during long conversations or complex tasks, and is related to the dilution of Transformer attention weights as sequence length increases.75% similarVerifiedThe Self-Attention mechanism of the standard Transformer requires global computation over the entire input sequence, making it inherently unsuitable for streaming scenarios where input is processed incrementally.73% similarVerifiedSet Transformer基于自注意力机制建模行向量交互，通过去掉位置编码保持排列不变性72% similarUnverifiedDecision Transformer通过自注意力机制直接建模跨越大时间跨度的因果关系，处理超长时序依赖72% similarUnverifiedImplementing a Transformer from scratch requires understanding numerical stability treatments such as why attention scores are divided by √d_k, and is a watershed task that distinguishes 'knowing how to use frameworks' from 'understanding the underlying principles'.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1002312API
curl https://kongchang.com/api/v1/knowledge/claims/1002312MCP
get_claim(id=1002312)