Unverified50% confidenceFactExact time
Learned resamplers (such as Q-Former) introduce additional parameters and attention computation overhead when compressing the number of tokens.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedMixed-precision quantization retains higher bit precision for key weights in attention layers while applying more aggressive compression to feed-forward layers with higher redundancy71% similarUnverifiedOn learned sparse retrievers, all expansion methods exhibited performance regression, corroborating concerns that query expansion may harm strong retrievers, while SAFE-QE still maintained a low hallucination rate in this scenario.69% similarUnverifiedStandard attention's computation grows quadratically with sequence length, whereas the low-rank bottleneck decomposes the transformation into two steps: first compressing to r dimensions and then expanding.68% similarUnverifiedThe ByteDance team's control experiments trained from scratch using the Qwen3-0.6B architecture show that all models using chunked compression exhibit periodic accuracy fluctuations, with the fluctuation period strictly equal to the compression step; the fluctuations still exist after removing RoPE or switching to evenly distributed gating weights; and the uncompressed full-attention baseline is completely normal.68% similarUnverified微调通过反向传播算法在预训练权重基础上继续更新参数，全量微调计算成本极高67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1014355API
curl https://kongchang.com/api/v1/knowledge/claims/1014355MCP
get_claim(id=1014355)