Unverified50% confidenceFactExact time
Gated Delta Net is a linear attention variant that uses an updatable state matrix instead of the key-value cache in traditional attention, with constant memory usage on long sequences, but the compression process is inherently lossy.
1
Sources
50%
Confidence
Long-term
Relevance
10/31/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedLinear attention-style architectures use a fixed-dimensional recurrent state to accumulate historical information; this state is a lossy compression of the context read so far, with a dimension far smaller than the original KV cache.69% similarUnverifiedBounded forgetting means the memory retrieval space should not grow without limit, and records that are no longer relevant should be retired on a schedule; unbounded memory causes short queries to face more and more long records, leading to hidden degradation of retrieval effectiveness.63% similarUnverified在自回归解码中，注意力层通常缓存所有历史 token 的 Key 和 Value（KV Cache），显存随序列长度线性增长，但每步只需追加、无需重算；SSM 层则只保留固定大小的状态。62% similarUnverifiedBW-Tree's Delta updates use CAS to point the mapping table pointer to a delta modification node, which has extremely low construction cost, but causes reads to require linear traversal of the Delta chain, hurting read performance.62% similarUnverifiedToken上下文窗口的遗忘效应是AI在连调阶段发明不存在接口的深层技术原因，当上下文窗口被填满时模型会开始遗忘早期信息61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1121143API
curl https://kongchang.com/api/v1/knowledge/claims/1121143MCP
get_claim(id=1121143)