Unverified50% confidenceFactExact time
The paper proposed conceptual frameworks such as the residual stream and attention head composition, using linear algebra language to analyze the attention mechanism inside Transformers.
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAttention Sinks were systematically described by Xiao et al. from MIT in the 2023 StreamingLLM paper, referring to the phenomenon where Transformer models allocate a large proportion of attention weights to the first 1–4 tokens at the beginning of a sequence.73% similarUnverifiedWhen text instructions precede image input, the Transformer's cross-attention mechanism causes the model to have already formed clear query vectors when encoding the image, preferentially activating visual regions semantically related to the target.73% similarUnverifiedThe paper proposes a method that simultaneously captures semi-local and global information flow features, focusing on attention heads that are highly correlated with hallucinated responses.71% similarVerifiedVision Transformer (ViT) introduced the self-attention mechanism from the NLP domain into image understanding, processing images by splitting them into patches and treating them as sequences.70% similarUnverifiedKuaishou's Kling uses a 3D causal attention mechanism to handle temporal frame relationships.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/995438API
curl https://kongchang.com/api/v1/knowledge/claims/995438MCP
get_claim(id=995438)