Unverified50% confidenceBenchmarkExact time
Redistributing the cross-image attention of 16 to 32 heads (keeping the total unchanged) causes synthetic-task accuracy to drop by 17 to 22 points and HPatches by 15 to 32 points, whereas random heads or attention-matched heads cause at most a 7-point drop.
1
Sources
50%
Confidence
Long-term
Relevance
11/17/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedUnder fixed compute, single-head attention scores 0.9 points lower than the optimal setting, and quality actually decreases when there are too many heads.76% similarUnverifiedThe cross-image comparison function is highly concentrated in a small number of attention heads, about 16 to 32 heads.75% similarUnverifiedResearch shows that when humans continuously perform high-attention monitoring tasks, their detection accuracy significantly declines after approximately 20–30 minutes.73% similarUnverifiedFor an unconstrained all-layer adapter, 21% to 51% of its gains are routed through the attention of the same set of key heads, compared with at most 7% for random heads.70% similarUnverifiedAfter applying DA, Gemma-4-31B reduced attention tokens by 52.0%, with an accuracy drop of only 1.27 percentage points.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1179119API
curl https://kongchang.com/api/v1/knowledge/claims/1179119MCP
get_claim(id=1179119)