[KongchangAI]
ConceptExplicit Breakpoints

Explicit Cache Breakpoints

A developer-controlled mechanism in large language model prompt caching that allows explicit marking of boundary points between stable prefixes and dynamic content within a prompt. This ensures precise cache hits and prevents dynamic content from invalidating cached entries, enabling more predictable and efficient use of the prompt caching system.

Source Articles