Opus 4.8 — cache-preserving mid-task system messages
LLMThe news. On May 28, 2026, Anthropic released Claude Opus 4.8, its new flagship. Alongside the headline agentic-coding gains, the announcement lists a quieter Messages API change: the model now accepts system entries mid-task without breaking the prompt cache. Anthropic described the behaviour, not the internals — but the behaviour is the interesting part, because mid-conversation instructions used to be one of the easiest ways to throw away a cache you had paid to build. Read the release →
Picture the metaphor first. Your kitchen line is prepped: vegetables chopped, sauces ladled, proteins portioned — your mise en place. That prep is the part you never want to redo mid-service. Now the expo clips a new order note onto the rail: "table 6 is allergic to garlic, hold it from here on." The note matters, and it arrives in the middle of service. The naive move is to treat the whole line as suspect and re-chop everything. The smart move is to clip the note to the rail and keep cooking — the prep on the line didn't change, so it stays exactly where it was.
A prompt cache is that prepped line. As the model reads your prompt it builds a KV cache — the keys and values for every token — and a prompt cache stores that work so the next turn can reuse it instead of re-reading the whole conversation. The catch is that the cache is a prefix cache: it is matched left-to-right by content, and it is only valid up to the first token that differs from what was cached. Append to the end and the cached prefix still holds. But change something in the middle — insert a new system message after a long conversation — and every token after the insertion point shifts, so a naive prefix cache has to recompute the entire downstream.
That is the "scrap the line and re-chop" failure. A long agent run can carry a large cached prefix; a single mid-task instruction that lands before it invalidates the lot, and you pay full prefill again — worse time to first token and full input price on tokens you had already cached. Opus 4.8's change is the "clip it to the rail" path: the system entry is accepted mid-task while the cached prefix is left intact. Anthropic published the outcome — the cache survives the insert — not the internal method; the prefix-cache picture above is the standard way to reason about why that outcome is worth having, not a disclosed description of how Opus 4.8 achieves it.
Naive insert vs. cache-preserving insert
For one mid-task system message dropped into a conversation with a long cached prefix:
| Path | What recomputes | Cached prefix reused |
|---|---|---|
| Naive insert (old behaviour) | the insert + everything after it | only the tokens before the insert (setup-dependent, illustrative) |
| Cache-preserving (Opus 4.8) | just the inserted message | the whole prior prefix (setup-dependent, illustrative) |
Walk one illustrative case. Say the conversation has a 9,000-token cached prefix, and you inject a 600-token system instruction, with 16,400 tokens of context that sit after the insertion point. The naive path recomputes the insert plus everything downstream — 600 + 16,400 = 17,000 tokens of fresh prefill — and reuses only the 9,000-token prefix, so cache reuse for that turn is 9,000 / 26,000 ≈ **35%** (illustrative). The cache-preserving path recomputes only the 600-token insert and reuses the rest, landing at 25,400 / 26,000 ≈ **98%** (illustrative). Same instruction, same result — the difference is entirely in how much warm cache survives the insert.
Goes deeper in: Agent Engineering → Cost & Latency → Prompt caching
Related explainers
- Claude Opus 4.8 — Parallel-subagent dynamic workflows — the other harness-level feature from the same release, on the orchestration side rather than the caching side
- Attention Once Is All You Need — Persistent KV cache across queries — a complementary angle on reusing KV across separate requests rather than within one conversation