Unverified50% confidenceFactExact time
The scaling factor (1/√d) in scaled dot-product attention only addresses the dimensionality problem, not the feature scale problem, so co-designing normalization and encoding is crucial.
1
Sources
50%
Confidence
Long-term
Relevance
11/2/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedMAE's limitation is that its decoder relies mainly on global self-attention and makes insufficient use of fine-grained structural priors such as local textures and edges.71% similarVerifiedThe core formula of Self-Attention is Attention(Q,K,V) = softmax(QKᵀ/√dk)V, where dividing by √dk is intended to mitigate the vanishing gradient problem caused by excessively large dot product values.70% similarUnverifiedThe principle of tiled processing is to split the feature map into sub-tiles along the spatial dimension for independent computation and then merge them; peak VRAM usage decreases linearly as the number of tiles increases, but overlapping sampling is required to avoid seams and the total computation increases slightly.69% similarVerifiedThe computational complexity of standard self-attention is quadratic with respect to sequence length, and achieving a context of one million tokens requires introducing techniques such as sparse attention, sliding window attention, and KV Cache compression.69% similarUnverifiedEntries with large attention weights do not necessarily contribute most to the output after matrix multiplication; weight and output contribution are two distinct quantities.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1126652API
curl https://kongchang.com/api/v1/knowledge/claims/1126652MCP
get_claim(id=1126652)