Unverified50% confidenceFactExact time
In decoding algorithms such as parallel sampling and beam search, one prompt generates multiple outputs, and each result requires storing the full sequence, causing the prompt to be stored repeatedly multiple times.
1
Sources
50%
Confidence
Long-term
Relevance
10/31/2026
First Seen
Sources
LLM Inference Bottlenecks and KV Cache: Core Mechanisms and Mainstream Optimization Approaches
bilibili先进编译实验室10/20/2026
Related Entities
Related Claims
UnverifiedThe runtime triage requires 12 parallel workers to perform triage, with results first kept in memory, written to PostgreSQL in batches of 50 with a final additional write at the end, and the dashboard reloaded after each batch save.69% similarVerifiedReranking uses a cross-encoder to score query-document pairs one by one, with accuracy far higher than vector similarity but at higher computational cost, so fine ranking is done only on the small number of candidates after coarse ranking, forming a two-stage architecture of coarse ranking plus fine ranking.69% similarUnverifiedMeasures to deal with duplicate chunks include exact deduplication via content hashing, near-duplicate clustering based on embedding similarity, establishing normalization policies, and weighting by source metadata at query time, none of which require changing the model.68% similarUnverifiedMulti-query retrieval uses an LLM to automatically generate 3–5 semantically equivalent but differently phrased variant queries for the same question, retrieves results for each separately, and takes the union to improve recall.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1121458API
curl https://kongchang.com/api/v1/knowledge/claims/1121458MCP
get_claim(id=1121458)