Conceptre-prefill
Re-prefill
In LLM inference, the process in which, when the KV cache is corrupted or the prefix is discontinuous, the model must recompute the forward pass over the input sequence to rebuild the cache, incurring additional latency and computation overhead.
Timeline (last 90 days)
Oct 21
Traditional methods reduce the KV cache by trimming or summarizing history through context engineering, which breaks the continuity of the trajectory prefix, causing the retained context to require re-prefill, increasing latency and potentially sacrificing task accuracy.
Unverified50%