Prompt Caching in 2026: Cut LLM Costs Across Claude, GPT, Gemini

Who this is for: developers building anything that resends a large, mostly-stable prompt on every request — agent harnesses, RAG pipelines, chatbots with long system prompts, coding assistants. If your token bill is dominated by re-processing the same 5,000-token system prompt every turn, prompt caching is the single highest-leverage change you can make this week.

Prompt caching lets an LLM provider skip re-processing the parts of your prompt that haven't changed since the last request, and charge you a fraction of the price for the tokens it reused. Done right, it cuts input costs by 80-90% on agentic and RAG workloads and shaves real latency off time-to-first-token. Done wrong — and this happens constantly — you pay a cache-write premium on every single request and never get a read back, because something as small as a timestamp in your system prompt broke the match. This guide covers how caching actually works, what to cache, what never to cache, and the differences between Anthropic, OpenAI, and Google's implementations as of mid-2026.

The one rule that explains everything

Prompt caching is a prefix match. The provider hashes your prompt content up to a marked breakpoint; if an earlier request wrote a cache entry with the identical byte sequence up to that point, the new request reads it at a steep discount. Change one byte anywhere before the breakpoint — reorder a JSON key, insert a live timestamp, add a tool — and every byte after that point is uncached, full price, no error, no warning.

This single fact explains nearly every caching bug you'll encounter. It also tells you the fix: get the order of your prompt right (stable content first, volatile content last) and caching mostly works itself out. Get the order wrong and no number of cache markers will save you.

On Anthropic's API, render order is fixed: tools → system → messages. A cache breakpoint placed on the last system block caches the tool definitions and system prompt together. This ordering is not configurable — it's baked into how the request is serialized before hashing, so design your prompt-building code around it rather than against it.

How cache_control works (Anthropic)

Anthropic's implementation is the most explicit of the three major providers: you place cache_control markers directly on content blocks.

{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "<50,000-token product knowledge base>",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "What's the return policy for opened electronics?" }
  ]
}

A marker can go on any content block — system text, tool definitions, or message content (text, image, tool_use, tool_result, document blocks). You get a maximum of 4 breakpoints per request. For the common case where you don't need fine-grained placement, a top-level cache_control on messages.create() auto-places the marker on the last cacheable block.

TTLs: 5 minutes vs 1 hour

"cache_control": { "type": "ephemeral" }                 // 5-minute TTL, default
"cache_control": { "type": "ephemeral", "ttl": "1h" }     // 1-hour TTL

The TTL resets on every cache hit — a cache entry read within its window gets its clock refreshed, so a chatbot with steady traffic every couple of minutes keeps one entry alive indefinitely on the 5-minute tier. The 1-hour tier exists for bursty traffic with gaps: a support bot that goes quiet overnight and picks back up in the morning.

Minimum cacheable prefix

Below a certain token count, a cache marker is silently ignored — no error, just cache_creation_input_tokens: 0 forever. The minimum is model-dependent:

This trips people up constantly when testing on a small system prompt against Opus — a 3,000-token prompt caches fine on Sonnet 4.5 and silently doesn't on Opus 4.8. If you're debugging "why isn't my cache working" and your prompt is under 4K tokens, check the model tier before anything else.

Pricing math: when caching actually pays off

Cache reads cost roughly 0.1x the base input price — a 90% discount. Cache writes cost 1.25x the base price for a 5-minute TTL, or 2x for a 1-hour TTL. That write premium means caching isn't free money on the first request — you need enough subsequent reads to amortize it.

The practical takeaway: a one-off request never benefits from caching, and a low-traffic endpoint that fires once every 20 minutes doesn't either — the write cost is dead weight. Caching pays off on anything hit more than twice within the TTL window, and pays off hugely on agentic loops where the same system prompt and tool definitions get resent 10-50 times in a single task.

Verifying it's actually working

Every response includes a usage object with three fields:

response.usage.cache_creation_input_tokens  # written this request, ~1.25x cost
response.usage.cache_read_input_tokens      # served from cache, ~0.1x cost
response.usage.input_tokens                 # uncached remainder, full price

If cache_read_input_tokens is zero across repeated requests with an identical prefix, something is silently invalidating the cache. This is the single most useful debugging habit for cache-heavy code: log these three fields on every call, not just token totals. Total prompt size is the sum of all three — if your agent's been running for an hour but input_tokens shows 4K, the rest was served from cache, and checking only input_tokens would make you think the prompt shrank.

What breaks caching (the silent-invalidator checklist)

These are boring bugs that cost real money because nothing ever throws an error.

Warning: switching models mid-conversation invalidates the cache completely — caches are scoped per model. If you're running a cheaper model for sub-tasks inside an agent loop, that's a deliberate cache miss, not a bug, but budget for it.

Not every parameter change costs you the whole cache, though. Anthropic's API has tiered invalidation:

So you can freely flip tool_choice per-request or toggle extended thinking without losing your tools+system cache — only tool-definition and model changes force a full rebuild.

Placement patterns that actually work

# Large shared system prompt across many requests — one breakpoint at the end
system=[{
    "type": "text",
    "text": LARGE_SHARED_PROMPT,
    "cache_control": {"type": "ephemeral"},
}]

# Shared prefix, varying question — breakpoint at the END of the shared part,
# not the end of the whole prompt (otherwise every request writes a fresh,
# never-read cache entry)
messages=[{"role": "user", "content": [
    {"type": "text", "text": SHARED_CONTEXT, "cache_control": {"type": "ephemeral"}},
    {"type": "text", "text": user_question},  # no marker — this varies every time
]}]

# Multi-turn conversation — marker on the last content block of the most
# recent turn; earlier breakpoints stay valid, so hits accrue incrementally
messages[-1]["content"][-1]["cache_control"] = {"type": "ephemeral"}

One easy-to-miss constraint on long agentic loops: each cache breakpoint only looks back 20 content blocks to find a prior match. A single turn with more than 20 tool_use/tool_result pairs will silently miss on the next request's breakpoint. Fix it by placing an intermediate marker every ~15 blocks in long turns.

There's also a timing gotcha for fan-out patterns: a cache entry only becomes readable once the first response begins streaming. If you fire five parallel requests with an identical prefix, none of them can read what the others are still writing — they all pay full price. Send one request, wait for the first token, then fire the rest.

Pre-warming for latency-sensitive first requests

If time-to-first-token on the very first user message matters (chat, voice), send a max_tokens: 0 request at startup. It runs the prefill, writes the cache at your breakpoint, and returns instantly with an empty content array — you pay the cache-write cost but no output tokens. Put the marker on the shared system prompt or tool definitions, never on the placeholder user message, or you'll cache the placeholder instead of what real requests need.

Provider comparison: Claude vs GPT vs Gemini in 2026

The practical difference that matters most day to day: OpenAI's caching is automatic and invisible — you get the discount if your prefix matches, with no API surface to manage, but also no way to force a longer TTL or verify hits beyond checking your bill. Anthropic gives you the most control (explicit breakpoints, TTL choice, per-request hit/miss visibility via usage). Google's explicit context caching is closer to a separate resource you create and reference by ID, which makes sense given Gemini's very large context windows — you're often caching hundreds of thousands of tokens of a single document, and paying to keep that alive for hours is worth the extra setup versus re-uploading it constantly.

If you're building a provider-agnostic layer — something like an AI gateway across multiple models — don't assume caching behavior transfers. A prompt structured for Anthropic's tiered invalidation rules won't automatically get the same discount from OpenAI's automatic caching, and switching providers on the same request path is itself a cache-buster no matter which one you're on.

What to cache — and what never to

Good caching candidates:

• Large, static system prompts (product docs, style guides, persona definitions)
• Tool/function definitions that don't change per user
• Few-shot examples baked into the system prompt
• Retrieved reference documents that are reused across many questions (a fixed knowledge base chunk, not per-query RAG results)
• The accumulated conversation history in a long-running agent session

Never cache:

• Anything containing a live timestamp, request ID, or session nonce
• Per-user personalization baked directly into the system prompt (move it to a later message instead)
• A prompt that changes from the first token on every request — there's no reusable prefix, so a cache marker only pays the write premium for zero reads
• Secrets or short-lived tokens — not because caching is insecure, but because caches are provider-managed and you have no fine control over eviction; keep credential injection out of anything long-lived (this is also why serious AI agent implementations route API keys through a vault or environment substitution, never through prompt text)

A concrete example: an agent harness done right

from anthropic import Anthropic

client = Anthropic()

STABLE_SYSTEM_PROMPT = """You are a code review agent. [... 6,000 tokens of
instructions, examples, and style guide that never change per request ...]"""

TOOLS = [
    {"name": "run_tests", "description": "Run the test suite", "input_schema": {...}},
    {"name": "read_file", "description": "Read a file from the repo", "input_schema": {...}},
]

def review_turn(conversation_history, new_user_message):
    messages = conversation_history + [{"role": "user", "content": new_user_message}]

    response = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=4096,
        tools=TOOLS,  # stable list, same order every call — position 0, cached
        system=[{
            "type": "text",
            "text": STABLE_SYSTEM_PROMPT,
            "cache_control": {"type": "ephemeral", "ttl": "1h"},
        }],
        messages=messages,  # last turn gets its own marker as the convo grows
    )

    print(
        f"cache_read={response.usage.cache_read_input_tokens} "
        f"cache_write={response.usage.cache_creation_input_tokens} "
        f"uncached={response.usage.input_tokens}"
    )
    return response

Notice what's absent: no datetime.now(), no per-request UUID, no conditional system-prompt assembly. The tool list is a static Python constant, not rebuilt per user. Everything volatile — the actual question — lives at the very end, after every cache breakpoint. This is the entire discipline; there's no cleverer trick beyond getting the ordering right and then watching the usage fields to confirm it's holding.

For teams building this kind of harness manually versus reaching for hosted agent infrastructure, it's worth reading up on agentic workflow patterns before committing to a manual tool-calling loop — a lot of the caching discipline above is handled for you by higher-level agent frameworks, at the cost of less visibility into what's actually cached.

FAQ

Does prompt caching reduce output token costs?

No. Caching only discounts input tokens — the parts of the prompt the model reads. Output tokens (what the model generates) are always billed at full price regardless of caching, on every provider.

How long does a prompt cache actually last?

On Anthropic's API it's either 5 minutes or 1 hour, and your choice resets on every cache hit — so a 5-minute cache read every 3 minutes never expires in practice. OpenAI's automatic caching is provider-managed with no published fixed TTL. Google's explicit context caching lets you set an arbitrary TTL when you create the cache object.

Why is my cache_read_input_tokens always zero?

Almost always a silent invalidator: a timestamp or random ID early in the prompt, non-deterministic JSON key ordering, or a tool/model that changes between requests. Diff the exact rendered prompt bytes between two calls — not just the visible text — to find the difference. See the silent-invalidator checklist above.

Is prompt caching worth it for a low-traffic app?

Only if you're hitting the same prefix more than twice within the TTL window. A single request pays the 1.25x-2x write premium with no reads to offset it — that's a net loss. If your endpoint gets hit rarely, either skip caching or extend the TTL to 1 hour and accept the higher write cost in exchange for a longer window to catch a second hit.

Can I cache different parts of a request separately?

Yes, on Anthropic's API — up to 4 breakpoints per request, each on a different content block (tools, system text, or specific message blocks). This lets you cache a stable system prompt at one TTL and a semi-stable retrieved document at another, while leaving the actual user question uncached.

Does switching models keep my cache warm?

No. Caches are scoped per model — switching from Sonnet to Opus mid-session, or routing a sub-task to a cheaper model, is a full cache miss on that request. If you're deliberately using a cheaper model for sub-tasks inside a larger agent, budget for that as an intentional miss rather than trying to preserve the cache across it.

Get this right the first time

Prompt caching is one of those optimizations that's cheap to get right and expensive to get subtly wrong — a single silent invalidator can mean paying full price for months without anyone noticing until the bill review. If you're building an AI product or agent and want a second pair of eyes on your caching setup, or want help architecting the prompt structure from scratch, grab a free 30-minute call through the contact form or message us on WhatsApp.

Sources

• Anthropic — Prompt Caching
• Anthropic — Pricing
• OpenAI — Prompt Caching
• Google — Gemini Context Caching