Unverified50% confidenceCautionExact time
Inference speed drops sharply once the context exceeds 128K, due to the ballooning KV cache size and the quadratic growth of attention computation time with sequence length
1
Sources
50%
Confidence
Long-term
Relevance
10/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedStandard attention's computation grows quadratically with sequence length, whereas the low-rank bottleneck decomposes the transformation into two steps: first compressing to r dimensions and then expanding.78% similarUnverified即便是支持128K上下文的模型，在文档长度超过32K时，中段信息的检索准确率往往出现明显下滑77% similarUnverified在推理阶段内存带宽往往比纯算力更早成为瓶颈，注意力机制需在每步生成时读取全部KV缓存，上下文长度增大时内存读写开销呈平方级增长76% similarUnverifiedMore aggressive quantization yields faster speed and smaller memory footprint but lower precision, while IQ3_XXS offers higher precision but requires close to 47 GB of memory and runs more slowly.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/969845API
curl https://kongchang.com/api/v1/knowledge/claims/969845MCP
get_claim(id=969845)