Unverified50% confidenceFactExact time
The cross-image comparison function is highly concentrated in a small number of attention heads, about 16 to 32 heads.
1
Sources
50%
Confidence
Long-term
Relevance
11/17/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedRedistributing the cross-image attention of 16 to 32 heads (keeping the total unchanged) causes synthetic-task accuracy to drop by 17 to 22 points and HPatches by 15 to 32 points, whereas random heads or attention-matched heads cause at most a 7-point drop.75% similarUnverifiedFor an unconstrained all-layer adapter, 21% to 51% of its gains are routed through the attention of the same set of key heads, compared with at most 7% for random heads.66% similarUnverifiedThe example model in the article has approximately 110,000 total parameters, with the Attention module accounting for 16,000 and the FFN accounting for 33,000.65% similarUnverifiedUsing /clear to start a new session for each task measured 19% and 16% of quota, versus 16% and 17% in the control group, a difference within the noise range, so no savings can be determined.65% similarUnverifiedAttention kernels often account for 30% to 60% of total compute during large model training and inference.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1179118API
curl https://kongchang.com/api/v1/knowledge/claims/1179118MCP
get_claim(id=1179118)