Feed Entities Podcasts Deep Dives Developers About

[KongchangAI]

Real-time AI knowledge base. Every claim is traceable to its source, with data freshness under 24 hours.

Powered by 240+ first-hand AI sources

Knowledge

Feed
Hot Entities
Podcasts

Developers

CLI
MCP Server
REST API
About

© 2026 KongchangAI. The freshest, most accurate AI knowledge

Unverified50% confidenceFactExact time

标准Softmax注意力在极长序列上存在注意力湮没（Attention Sink）问题，催生了Sliding Window Attention、RoPE外推、ALiBi等改进方案

1

Sources

50%

Confidence

Long-term

Relevance

7/10/2026

First Seen

Sources

GLM-5.2发布：开源编程能力首次逼近闭源旗舰模型

bilibili五里墩茶社6/22/2026

Related Claims

UnverifiedTransformer架构的注意力机制中，序列长度增加时早期token的注意力得分在softmax归一化后被后续token稀释，形成注意力沉没效应78% similar Unverified流式处理的解决方案包括因果注意力掩码、chunked attention和look-ahead缓冲76% similar VerifiedMulti-Head Latent Attention（MLA）是DeepSeek-V2提出的注意力机制75% similar Unverified标准Attention的显存消耗随序列长度平方增长，Flash Attention将其压缩到线性增长74% similar Unverified新内核为滑动窗口注意力（SWA）层提供双输入支持，并引入 attention sinks 机制74% similar

Cite This Claim

Stable URI

https://kongchang.com/claim/453428

API

curl https://kongchang.com/api/v1/knowledge/claims/453428

MCP

get_claim(id=453428)