GPT-6 Prompt Caching Cuts Agent Input Costs by Up to 90%: What to Change in Your Prompts This Week

OpenAI shipped a change this week that cuts the price of cached input tokens on GPT-6 by up to 90%, and I’d bet most teams haven’t read past the announcement. It lives in the boring part of the docs. It’s called prompt caching, and for anyone running agents in production it’s the difference between an API bill that stings and one you barely notice.

Here’s the part that matters: the discounts apply automatically now, and shared prefixes stay eligible for reuse across requests within a 30-minute window. If your agent sends the same system prompt, tool definitions, and context on every call, which persistent agents do by definition, most of your input tokens can be served from cache instead of being reprocessed at full price.

What actually changed

Three things, and they compound.

First, the economics. Cached input tokens cost up to 90% less than fresh ones. OpenAI also cut model prices across the board the same week (GPT-6 Sol dropped to $2 per million input tokens and $10 output; the budget Luna tier is at $0.10/$0.50). Stack the two and the arithmetic gets silly. A million cached input tokens on Sol runs about 20 cents instead of two dollars.

Second, eligibility got wider. Prefixes reused within a 30-minute window qualify. That window matches how real agents work, where the same conversation context gets resent on every tool call for hours.

Third, and this is the genuinely new part: observability. A Prompt Caching Dashboard shows what share of your input is served from cache over time. When a cache miss happens, a diagnostics tool tells you why it missed and how many tokens it cost you. Model changed. Tools changed. Settings changed. No more squinting at a Tuesday bill spike and guessing.

Why agents are the big winners

A chatbot sends maybe a system prompt and one message. An agent sends its whole kit on every single turn: instructions, tool schemas, retrieved documents, conversation history. As those loops get longer, input tokens stop being a rounding error and start being the line item.

GitHub ran the numbers on this exact stack. Their CPO says Copilot cut the share of prompt tokens needing fresh processing by more than 50% across billions of requests. That’s with codebase context and long sessions, the same shape of workload most production agents have.

The kicker, though, is that cache behavior is something you design for, not something that happens to you. Two agents with identical workloads can land wildly different hit rates depending on how their prompts are assembled.

The habits that quietly break your cache

This is where I see teams lose money without realizing it.

Rearranging the top of the prompt. Caching matches from the front. If anything near the start of your prompt changes between requests, a timestamp, a per-request ID, a randomized few-shot example, the prefix match dies right there and everything after it gets billed fresh. Keep the static material at the top and the dynamic stuff at the bottom.

Deleting tool definitions between calls. The instinct is to slim the request by removing tools you’re not using this turn. That rewrites the prefix and invalidates the cache. OpenAI’s guidance is to use allowed_tools or tool_choice to constrain what the model can reach while keeping the definitions stable in the payload.

Changing reasoning effort carelessly. Normally, changing settings mid-conversation busts the cache. The update adds a way around it: a configuration_update message lets you shift reasoning effort between responses without starting over. If you’ve been pinning effort to one level for cache reasons, you don’t have to anymore.

There are also explicit cache breakpoints now, so you can mark which parts of a long prefix you want treated as reusable rather than hoping the automatic prefix detection catches it.

The math, concretely

Say you run a content pipeline that fires 500 agent sessions a day, each sending roughly 40,000 tokens of shared instructions, tool schemas, and reference material, plus maybe 2,000 tokens that are genuinely unique per request.

At Sol’s list price of $2 per million input tokens, the fresh-everything approach costs you about $42 a day on input alone. Serve the shared 40K from cache at the 90% discount and it drops to about $4.40. Multiply across a month and you’re looking at roughly $1,260 versus $130. Same model, same outputs, same workload. The only thing that changed was whether the prompt was arranged to hit the cache.

Those are clean round numbers, to be fair, and your real workload won’t be this tidy. But the shape holds: when your input is mostly shared context, and agent input usually is, the cache decision dominates the bill.

What to do this week

1. Open the Prompt Caching Dashboard and look at your actual hit rate. If it’s low, you found money.
2. Audit your prompts for anything volatile near the top: timestamps, request IDs, shuffled examples. Move it all below the stable prefix.
3. Stop deleting tool definitions between calls. Use allowed_tools to gate access instead.
4. Set explicit cache breakpoints on your long shared prefixes so reuse doesn’t depend on luck.
5. Recheck the dashboard after a week. The diagnostics tool will name whatever is still costing you.

The quiet lever

Most of the AI cost conversation is about model choice: pay for Astra-tier depth or drop down to Sol or Luna. That’s a real decision and the price cuts this week made the mid-tier genuinely attractive. But model choice is a coarse lever. Caching is the fine one, and it’s the one almost nobody teaches.

The teams that got 50%+ reductions at GitHub-scale didn’t invent anything. They just stopped paying twice for the same tokens. Given that the fix is mostly rearranging a prompt, this might be the highest hourly rate you’ll ever earn.