OpenAI Improves GPT-6 Prompt Caching: Faster and Cheaper Agents with Up to 90% Discount on Cached Inputs

OpenAI has announced an upgrade to its prompt caching mechanism in the API, primarily targeting the upcoming GPT-6 model. The improvement is designed to make agent applications built on large language models run faster and at lower cost. The core change: higher cache-hit rates by default, allowing more input tokens to benefit from cached input discounts of up to 90%.

What Problem Does Prompt Caching Actually Solve?

In real-world agent or conversational applications, each API call often carries large amounts of identical context — system prompts, tool definitions, few-shot examples, and conversation history. This content remains virtually unchanged across multiple turns of interaction, yet if treated as entirely new input each time, it generates redundant computation costs and charges.

The idea behind prompt caching is straightforward: cache these recurring input prefixes on the server side so that subsequent requests hitting the cache are billed at a significant discount rather than full price. The "up to 90% cached input discount" OpenAI mentions means that cached input tokens can cost as little as one-tenth of the standard price — a substantial saving for high-frequency use cases.

Under the hood, prompt caching relies on the KV Cache (Key-Value Cache) mechanism. When a large language model processes input, it computes Key and Value matrices for each token as part of the attention mechanism — one of the most computationally expensive steps during inference. The essence of prompt caching is storing the KV Cache results for input prefixes on the server, and when a subsequent request's prefix matches cached content, those intermediate results are reused directly, skipping redundant computation. A cache hit therefore isn't just a billing discount strategy — it reflects genuine compute savings. The server truly does less work, which is why hitting the cache delivers both lower latency and lower pricing simultaneously. It's worth noting that cache matching typically requires an exact prefix match (down to the token sequence), which is the fundamental reason developers need to place stable content at the front and keep its order consistent.

What Does a Higher Default Cache-Hit Rate Mean?

The key phrase in this upgrade is "higher cache-hit rates by default." Previously, developers who wanted to fully leverage caching often had to carefully engineer their prompt structure — ensuring reusable content was placed in the prefix and maintained in a stable order. OpenAI now says the improved system can push more input tokens into the cached range by default, without requiring that level of manual optimization.

This delivers value on two levels:

• Cost: More tokens benefit from the discount, directly lowering the per-call cost. For production-grade agents with massive daily call volumes, the cumulative savings can be substantial.
• Performance: Cache hits don't just save money — they also reduce latency from redundant processing, making agents more responsive. This is especially valuable for multi-step reasoning workflows and agents that call tools frequently.

Why This Matters Especially in the Agent Era

AI applications are shifting from single-turn Q&A toward multi-step, long-context agent workflows. A typical agent task might involve dozens of model calls, each carrying large and heavily overlapping contexts. In this paradigm, caching efficiency directly determines an application's economics and viability.

By tying this caching upgrade to GPT-6, OpenAI is sending a clear signal: the next generation of models isn't just more capable — the underlying infrastructure is being specifically optimized for large-scale, long-running agents. Improvements in cache-hit rates, latency, and cost together lower the barrier to building complex agent systems.

Agent applications typically follow a ReAct (Reasoning + Acting) or similar "think–act–observe" loop architecture, where each cycle triggers a new model call and the context grows as tool results accumulate. Consider a complex task involving search, code execution, and database queries: completing it might require 20 to 50 model calls, with each call's input potentially reaching tens of thousands of tokens — and system prompts plus tool definition schemas could account for 60–80% of that as fixed content. At this call density, improving the cache-hit rate from 60% to 80% can reduce total per-task costs by more than 30%. This means caching infrastructure efficiency is becoming a critical variable in determining the commercial viability of agent applications — not merely a nice-to-have optimization.

Practical Tips for Developers

Although OpenAI emphasizes that higher hit rates are now available "by default," teams pursuing aggressive cost optimization should still keep a few things in mind:

• Place stable, unchanging content (system prompts, tool schemas) at the beginning of your prompt, and variable content at the end, to maximize prefix reuse.
• Monitor your actual cache-hit rates and the proportion of tokens covered by the discount to verify that optimizations are genuinely taking effect.
• When designing high-frequency agent loops, factor cache savings into your cost estimation models.

It's worth noting that this information comes from a brief announcement posted by OpenAI on social media. Specific pricing details, cache TTL (time-to-live), and the full capabilities of GPT-6 should be confirmed against official documentation when available.

Summary

OpenAI's improvements to GPT-6 API prompt caching are fundamentally an infrastructure optimization for the agent era: by delivering higher default cache-hit rates, more input tokens can benefit from discounts of up to 90%, while response latency is also reduced. For production applications that rely on large amounts of repeated context, this is a practical upgrade that simultaneously improves both cost efficiency and user experience.