Unverified50% confidenceFactExact time
ALiBi directly superimposes a linear penalty term proportional to distance onto attention scores, achieving a clean implementation that is friendly to length extrapolation
1
Sources
50%
Confidence
Long-term
Relevance
9/21/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedALiBi 不引入显式位置编码，而是在注意力分数上叠加与相对距离成线性比例的负偏置项75% similarUnverified相对位置编码（如 RoPE、ALiBi）会对距离较远的 token 对施加注意力衰减偏置，序列边界位置拥有更多邻近token的注意力积累优势72% similarUnverified原始 Self-Attention 的计算量随序列长度呈平方级增长，Anthropic 通过优化位置编码（如 ALiBi）等技术手段拓展了可用上下文68% similarUnverifiedSolutions for streaming processing include causal attention masking, chunked attention, and look-ahead buffering63% similarUnverifiedCIoU损失在GIoU基础上引入长宽比一致性惩罚项，同时优化重叠面积、中心点距离与形状相似性61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/939622API
curl https://kongchang.com/api/v1/knowledge/claims/939622MCP
get_claim(id=939622)