What is a prompt-caching mis-integration?

It is application code that breaks a provider's rules for enabling, configuring or accounting for prompt caching, so the cache stops hitting or the usage numbers are wrong. In the CacheLint study, 90% of 120 real cases raised no error at all, and the only trace was extra cost or latency.

Why does a timestamp in the system prompt break caching?

Providers match the prompt byte by byte from the first token, so a value that changes on every call makes the prefix differ before any stable content is reached. The fix seen in real projects is to move the changing value out of the cached span, or coarsen it, for example to the date instead of the second.

How does CacheLint decide that a model supports caching?

For capability conditions it compares what the code assumes (for example a model-name check) against a dated provider capability table, not an LLM. The table is dated because provider rules change after the code ships, so correct code can become wrong without any edit.

Agent· 2026-10-28

CacheLint paper — Prompt-cache preconditions — What does it mean?

Jargon

Prompt caching: The provider keeps the computed state of a prompt prefix and reuses it when a later request starts with the exact same bytes, charging cached tokens at a much lower rate. See Prefix Caching for how the matching works.
Prefix stability: The requirement that the start of the prompt is byte-for-byte identical from call to call. One changed character early in the prompt invalidates the cache from that point onward.
Cache marker: A provider-specific request field that asks for caching, such as cache_control, cachePoint, cachedContent or a TTL setting. Each provider accepts its own shapes.
TTL: Time to live: how long a cached prefix stays available. A window that is too short means a cache miss even when the prompt is identical.
Mis-integration: The paper's term for application code that violates a provider's requirements for enabling, configuring or accounting for prompt caching.
Provider capability oracle: A dated lookup table of which provider and model support which caching features. It is dated because providers change their rules after your code ships.
CodeQL: A code-query tool that finds places in source code matching a pattern. CacheLint uses it to locate every cache call site.

The news. On October 26, 2026, researchers from Wake Forest, Griffith, NTU, UNSW and SUSTech posted a study of prompt-caching bugs in LLM agents. They labeled 120 caching pull requests from 10 popular open-source agent systems, built CacheLint, and ran it on 571 agent projects, where it found 32 more mis-integrations. Read the paper →

Picture the print shop. Your agent sends the same long booklet on every turn: instructions, tool descriptions, then the conversation so far. The shop only reuses a shelved copy if your booklet matches it from page 1, character by character. So a clock reading stamped on page 1 turns every later page into a mismatch, and the shop reprints the whole booklet at full price. The model still answers correctly, which is why nobody notices.

The paper sorts the real fixes by who reads or writes the cache wrongly. 81% of the 120 defects are on the request side (97 cases): the app builds the request so the cache cannot hit. The other 19% are on the read side (23 cases): the cache works, but the app misreads the receipt, for example by hardcoding cache-creation tokens to zero. On the request side, two families dominate. In provider compatibility, an app turns caching on by model name, so a renamed model silently loses it; this is what happened in the cline project when a Gemini model ID changed. In prefix stability, a volatile value such as a timestamp or a random call ID sits inside the cached span.

CacheLint turns this into a small formal model. Every bug is a pair: a call site, plus a condition that must hold at that site and is not guarded. The site is a line that attaches a marker (like cache_control) or reads a usage field (like cached_tokens). The condition belongs to one of three families: prefix stability, provider capability, or accounting safety. CodeQL finds the sites. Then the check depends on the family: capability conditions are checked against a dated provider table, not by a language model, because the answer lives outside your code and can change after release (the paper notes that when Bedrock added a one-hour TTL, previously correct code silently became defective). Stability and accounting conditions are checked by an LLM that reads the code around the site and decides whether a guard exists.

Find the call siteCodeQL locates each cache marker or usage read

Pick the conditionstability, capability or accounting

Check itdated oracle, or an LLM reads the surrounding code

Warn with a fix hintonly if no guard discharges the condition

Root cause	Real example from the study	What the owner sees	Fix pattern
Model-name gating	cline: only IDs starting with `claude` got `cache_control`, so a renamed Gemini model lost caching	Bill rises, no error	Ask a capability lookup, not a hardcoded name
Volatile value in the prefix	goose: a live timestamp in the system prompt	Cache never hits	Move the changing value out of the cached span
Prefix reassembly	hermes-agent: the system prompt rebuilt every turn, reported as roughly 75% more cost	Higher cost	Rebuild the prefix in one fixed order
Nondeterministic serialization	openclaw: map order differs per instance	Identical content, different bytes, miss	Sort keys so the bytes are canonical
Cross-turn mutation	openclaw: pruning old images rewrote the already-cached history	Cache rebuilt instead of read	Do not edit cached history mid-session
Usage miscount	cline: cache-creation tokens hardcoded to zero	Meter says caching is off while it works	Parse and sum the provider's fields correctly

Here is why one stray timestamp matters, with illustrative numbers. Take an agent task of 30 model calls, each carrying a 20,000-token stable prefix, and say a cached token costs one tenth of a normal input token (the paper cites roughly one-tenth). With the prefix broken on every call, the task pays for 30 × 20,000 = 600,000 full-price tokens. With a stable prefix, the first call writes the cache at full price (20,000) and the other 29 calls read it at one tenth: 29 × 20,000 × 0.1 = 58,000, for a total of 78,000 token-equivalents. That is about 7.7× cheaper for the same answers (illustrative). Real cache-write surcharges and TTL expiry would move the numbers, but the shape holds: the loss is large and invisible in the output.

Two limits keep this honest. First, CacheLint is a linter, not a proof: it reached 0.71 precision and 0.78 recall on the 120 labeled cases, and 0.86 precision on the 32 new findings across 571 projects, with 21 developer confirmations and 18 upstream fixes, including one in Anthropic's official SDK. Second, two root causes sit outside its grammar: cross-turn mutation and prefix reassembly compare prefixes across different calls, and one program state cannot show that. Those still need a runtime check, such as watching the cache-hit rate in your traces.

Goes deeper in: Agent Engineering → Cost & Latency Engineering → Prompt Caching

Related explainers

Cache-preserving mid-task system messages — a provider-side feature that adds a message without breaking the cached prefix.
Prefix-cache state and agent irreproducibility — what else a cache can change besides your bill.

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based