Unverified50% confidenceFactExact time
Existing structured pruning methods usually assign and rank importance weights at the MLP block or attention head level, with relatively coarse granularity, which loses the information about differences in importance among different neurons within an MLP block.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedPruning compresses model size by removing neurons or attention heads that contribute little to the output.78% similarUnverified研究表明注意力层主要负责信息的路由和聚合，而MLP层更像是知识的存储库，负责非线性变换和信息的深度加工71% similarUnverifiedToken pruning refers to directly discarding some visual tokens based on importance scores, with a high compression rate but irreversibility; token merging instead aggregates similar tokens into representative vectors, preserving information while reducing sequence length.70% similarUnverifiedDense attention layers require a complete KV Cache, whereas linear layers rely on hidden states.68% similarUnverifiedSpeedup ratios for attention operators are often highly sensitive to specific shapes and scenarios, with the decode phase being memory-access-intensive and the prefill phase being compute-intensive.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1037989API
curl https://kongchang.com/api/v1/knowledge/claims/1037989MCP
get_claim(id=1037989)