Session Affinity
A routing strategy that pins consecutive requests of the same session to the same provider in order to maximize prompt cache utilization and reduce the cost and latency of multi-turn conversations.
Timeline (last 90 days)
The core logic of session affinity is to send each turn of a request to the same provider that served the previous turn, so as to leverage that provider's cache.
Session affinity takes effect by default on text and chat requests and requires no additional configuration.
Session affinity binds a session to a single provider, so when that provider experiences fluctuations, the continuity of the session may be affected.
Coding agents (such as Claude Code, Codex, and Cursor) are typical multi-turn, long-context scenarios, and the cache reuse benefit brought by session affinity is most pronounced in these scenarios.
The arXiv paper Vesta proposes decoupling 'session-level KV cache affinity' from 'request-level execution placement', using queuing theory to trade off between cache locality and queue congestion.
All Facts (5)
The core logic of session affinity is to send each turn of a request to the same provider that served the previous turn, so as to leverage that provider's cache.
50%UnverifiedSession affinity takes effect by default on text and chat requests and requires no additional configuration.
50%UnverifiedSession affinity binds a session to a single provider, so when that provider experiences fluctuations, the continuity of the session may be affected.
50%UnverifiedCoding agents (such as Claude Code, Codex, and Cursor) are typical multi-turn, long-context scenarios, and the cache reuse benefit brought by session affinity is most pronounced in these scenarios.
50%UnverifiedThe arXiv paper Vesta proposes decoupling 'session-level KV cache affinity' from 'request-level execution placement', using queuing theory to trade off between cache locality and queue congestion.
50%