Unverified50% confidenceConceptExact time
Learned sinks are trainable absorption positions introduced into the attention softmax, used to stabilize the attention distribution in long-context or streaming generation.
1
Sources
50%
Confidence
Long-term
Relevance
10/21/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe learnable routing function differs from fixed sparse attention (such as local windows and fixed-stride jumps) by allowing the model to learn during training where it should attend.64% similarUnverifiedIn-context learning in the tabular domain typically relies on attention mechanisms that let the prediction sample interact directly with the training samples in the context, equivalent to soft retrieval weighting at inference time.64% similarUnverifiedLatent Compaction training is divided into two stages: the first stage fits the attention output and log normalizer of the full cache, and the second stage jointly optimizes the entire cache through end-to-end distillation.63% similarUnverified少样本示例（Few-shot Prompting）的工作原理基于Transformer注意力机制，无需更新模型权重，被称为上下文学习（In-context Learning）63% similarUnverifiedIn the early stage of hybrid architecture training, the recurrent layer rapidly masters retrieval capability and suppresses the signal for the attention layer to learn the same function, causing the attention layer to be unable to effectively learn retrieval in most cases and impairing overall performance.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1081600API
curl https://kongchang.com/api/v1/knowledge/claims/1081600MCP
get_claim(id=1081600)