Unverified50% confidenceFactExact time
AttSVD introduces a per-matrix energy rule that independently determines sizes for logit space and attention quality, dynamically adjusting the compression ratio.
1
Sources
50%
Confidence
Long-term
Relevance
10/7/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe basis used by AttSVD for low-rank compression is not fixed but is derived from each prompt's own attention geometry, retaining only the directions actually read by attention via online per-prompt truncated SVD.70% similarUnverifiedLinformer 将注意力矩阵低秩近似为 O(n) 复杂度，FlashAttention 通过重排计算顺序降低 GPU 内存带宽占用，Longformer 引入滑动窗口注意力59% similarUnverifiedSVM finds the optimal hyperplane by maximizing the classification margin, and the kernel trick can implicitly map data to a high-dimensional space to handle non-linearly separable problems.59% similarVerifiedFor a model with L layers, H attention heads, and per-head dimension d, caching n tokens requires VRAM of 2 × L × H × d × n × (precision bytes).58% similarUnverifiedIn the standard attention mechanism, the Query matrix and Key matrix perform matrix multiplication to obtain an n×n attention score matrix, with a computational complexity of O(n²·d).57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/983259API
curl https://kongchang.com/api/v1/knowledge/claims/983259MCP
get_claim(id=983259)