P99 Latency
A percentile metric in a latency distribution indicating that 99% of requests have latency no greater than that value, used to measure tail latency.
Timeline (last 90 days)
What online decision pipelines are truly sensitive to is P95 or P99 latency, because timeouts and tail latency often determine whether a system needs to degrade or set fallback rules.
Latency perception depends heavily on context, and there is no universal number of milliseconds; the engineering goal should target a comfortable range of about 1.5 seconds; consistency matters more than peak speed, and attention should be paid to P99 latency rather than peak speed.
The core metrics for evaluating vector indexes include recall (Recall@k), P99 latency, QPS (throughput), index build time and memory.
In agent scenarios, a single complex task often requires multiple LLM calls chained in series, making control of P99 latency more practically meaningful than average latency.
For interactive search, P99 latency typically needs to be kept within a few hundred milliseconds.
All Facts (5)
What online decision pipelines are truly sensitive to is P95 or P99 latency, because timeouts and tail latency often determine whether a system needs to degrade or set fallback rules.
50%UnverifiedLatency perception depends heavily on context, and there is no universal number of milliseconds; the engineering goal should target a comfortable range of about 1.5 seconds; consistency matters more than peak speed, and attention should be paid to P99 latency rather than peak speed.
50%UnverifiedThe core metrics for evaluating vector indexes include recall (Recall@k), P99 latency, QPS (throughput), index build time and memory.
50%UnverifiedIn agent scenarios, a single complex task often requires multiple LLM calls chained in series, making control of P99 latency more practically meaningful than average latency.
50%UnverifiedFor interactive search, P99 latency typically needs to be kept within a few hundred milliseconds.
50%