Unverified50% confidenceFactExact time
Attention Sinks 来自麻省理工学院 2023 年的 StreamingLLM 研究
1
Sources
50%
Confidence
Long-term
Relevance
7/16/2026
First Seen
Sources
Related Claims
Unverified标准Softmax注意力在极长序列上存在注意力湮没（Attention Sink）问题，催生了Sliding Window Attention、RoPE外推、ALiBi等改进方案72% similarUnverified新内核为滑动窗口注意力（SWA）层提供双输入支持，并引入 attention sinks 机制70% similarUnverifiedSolutions for streaming processing include causal attention masking, chunked attention, and look-ahead buffering66% similarVerifiedMulti-Head Latent Attention (MLA) is an attention mechanism proposed by DeepSeek-V2.65% similarUnverified标准Attention的显存消耗随序列长度平方增长，Flash Attention将其压缩到线性增长65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/527874API
curl https://kongchang.com/api/v1/knowledge/claims/527874MCP
get_claim(id=527874)