Foreign KV caches paper — Substitution vs augmentation of a borrowed cache — What does it mean?

The news. On October 15, 2026, researchers from MIT and IBM Research posted a paper that asks what a frozen model will accept from another model in place of its own KV cache. All experiments use one reader (Qwen2.5-1.5B, 28 layers) on a factual-recall probe built from SQuAD, so the authors present it as a characterization, not a leaderboard result. Read the paper →

Picture two students who have both studied the same passage. One of them, the sender, wrote careful margin notes. The reader sits an exam with the answer line blacked out. The tempting shortcut is to take the colleague's notes and slide them into your own binder, in place of the notes you wrote. That is substitution, and it is exactly what a frozen reader is asked to do when a foreign KV cache is installed in its slots.

In the tested swaps, the binder rejects them. A frozen transformer appears to read correctly only from caches that sit very close to the cache it would have written itself, a region the authors call its tolerable set. The surprise is how thin that region is, and that the usual ruler for closeness, cosine similarity, does not measure it.

Closeness is not usability

The authors first perturb the reader's own cache by a calibrated amount. With the answer masked, the reader's own cache scores the floor, 0.180. At cosine 0.9999 the score is still 0.180, at 0.999 it drops to 0.120, and at 0.995 it falls to 0.005, far below a cache that carries no information at all. Then they compare that to a real cache the same model wrote on the unmasked passage. It sits at cosine 0.9949, further from the reader's own cache than the noise, yet it scores 0.445. At matched cosine the ordering is reversed: the genuine cache is usable and the noisy one is not. The property that matters looks more like provenance (could a real forward pass have produced this?) than distance.

Two more results rule out the obvious explanations. Corrupting a single layer of 28, while the other 27 stay bit-identical, drops the score from 0.665 to 0.030, and corrupting all 28 does no worse, so in this sweep one corrupted layer was nearly as damaging as all of them. And handing the reader the nearest genuine key from a library of 6,000 keys it wrote on other passages scores 0.000, so even a real key from the right model is not enough.

Why: attention is a vote, not a distance

Every cache enters the reader through one operation, softmax attention, where keys are scored against each other. Whether a note is read depends on which keys win each vote and by what margin, never on how close the cache is in any norm. A shift too small to register as distance can flip a vote, after which the reader looks at the wrong part of its context. A modest context holds tens of thousands of these votes (one per query, per head, per layer), and the cache must survive all of them. This also explains why compressing a cache with the same model works: the reader is judging a cache it wrote itself, so it asks the question from inside its tolerable set.

In the study metaphor, your notes and the colleague's can look identical on the page and still make a different note win the "which line answers this question?" contest. The cosine ruler averages over directions the reader ignores and directions it resolves sharply, and reports neither.

What works instead, in numbers

Hold three things fixed: the reader (Qwen2.5-1.5B), the probe (200 SQuAD items), and the two anchors. The floor of 0.180 is 36 of 200 items correct, and the native-cache reference of 0.665 is 133 of 200. The best of the eight substitution methods reaches 0.200, which is 40 items, nominally 4 items above the floor. The same reader with the sender's cache simply appended as extra context, and zero trained parameters, reaches 0.555, which is 111 items, 75 items above the floor. Adding rank-16 LoRA on the queries reaches 0.695, or 139 items. At n = 200 that is six items above the reference, so the paper reports it as reaching the reference, not beating it.

There is a caveat on that gap. Augmentation also gives the reader strictly more to attend over, so part of the gain is not placement alone. The closest matched comparison is a cache that carries the answer: 0.445 when substituted into the slots versus 0.555 when supplied as context. In the study metaphor, keep your own notes and put the colleague's on a side table, then learn which questions to ask them.

Many senders, and the limits

A multi-agent system rarely has one sender. When the reader attends over the union of N senders' caches, with only one having read the answer passage, plain attention over the union falls below the floor by four senders, while the query-adapted reader decays gradually from 0.785 at N = 1 to 0.395 at N = 8. The paper did not measure where that reader places its attention, so it does not claim selectivity as the cause.

Read the scope carefully. Everything is one reader, one task type (short factual recall), and a same-checkpoint setup, so it shows placement within one model, not communication between two. The authors also do not claim that a bridge fitted against the reader's own caches is blocked; they did not test one. Across model families the augmentation arm failed at dimension matching. The practical lesson for context engineering and cache-sharing designs is narrower than "caches can be shared": a borrowed cache should be offered to the reader as context, not installed in place of its own, and a cache-sharing result reported only as cosine or reconstruction error says little about whether the reader can use it.

Goes deeper in: LLM Internals → KV Cache → The KV Cache Solution