Unverified50% confidenceFactExact time
The attention sink phenomenon was systematically described by Xiao et al. from MIT in the 2023 StreamingLLM paper: Transformers allocate a large proportion of attention weights to the 1st–4th tokens at the beginning of the sequence.
1
Sources
50%
Confidence
Long-term
Relevance
8/31/2026
First Seen
Sources
Related Entities
Related Claims
Unverified线性注意力思路由 Katharopoulos 等人在 2020 年论文《Transformers are RNNs》中系统化73% similarUnverifiedDai等人在2022年发表了论文《Knowledge Neurons in Pretrained Transformers》，通过定位和编辑特定神经元可以修改模型对特定事实的记忆64% similarUnverified《Attention Is All You Need》论文在发表初期并未全面碾压所有基线，却奠定了Transformer时代的架构基础64% similarVerifiedTransformer最初由Vaswani等人在2017年的论文《Attention Is All You Need》中提出，用于机器翻译任务60% similarUnverifiedThe field of mechanistic interpretability was largely founded by the Anthropic research team (especially Chris Olah) around 2021 through the "Transformer Circuits" series of papers.59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/828094API
curl https://kongchang.com/api/v1/knowledge/claims/828094MCP
get_claim(id=828094)