Unverified50% confidenceFactExact time
When KV cache is reused, cached intermediate activations may produce different logit outputs for the same input when the batch size or sequence concatenation method changes.
1
Sources
50%
Confidence
Long-term
Relevance
9/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe KV Cache mechanism can cache repeated prefix content, and the more concise the system prompt, the higher the cache hit rate.77% similarUnverifiedPrompt caching technology stores KV matrices in high-speed memory, and when the next request contains the same prefix content, it reads directly from the cache, skipping recomputation.68% similarUnverifiedDecode 阶段属于访存密集型操作，每生成一个新 token 需从显存读取全部历史 token 的 KV Cache66% similarUnverified输入Token可以在prefill阶段并行处理，而输出Token必须在decode阶段逐个串行生成65% similarUnverifiedPrompt Caching缓存的是输入而非输出，缓存LLM的输出几乎没有意义63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/916504API
curl https://kongchang.com/api/v1/knowledge/claims/916504MCP
get_claim(id=916504)