Cartridges match in-context learning once retrieval is real — KV cartridges vs parametric fine-tuning — What does it mean?

The news. On September 15, 2026, a paper titled Where Should a Document Live: Context, Representations, or Parameters? posted to arXiv. It runs a controlled comparison of representation-based (KV-cache) and parametric (fine-tuning) adaptation across five knowledge-intensive benchmarks, holding the knowledge task and the storage budget fixed so the methods can be read against each other instead of against their own papers. The headline result: in the oracle setting, Cartridges (KV) are the most accurate injection method at nearly every storage budget, ahead of parametric methods by 10 points. Read the paper →

Picture the bench. The manual lying open takes almost the whole surface, and it is the thing you actually read — nobody misquotes a page they can see. The index cards are what you wrote after studying it: far smaller, and they still answer the question. The shrunk photocopy is those same cards squeezed until the print starts to blur. Memorising the manual costs nothing at bench time and a great deal everywhere else. The paper's question is not which of these is best in the abstract — it is which one wins when the bench is a fixed size.

In model terms the bench is a storage budget and the four options are four places to put a document. In-context learning leaves it in the prompt, where context is the scarce resource. A Cartridge trains a compact KV cache that stands in for the document. Compaction takes a KV cache that already exists and compresses it. Parametric adaptation — full fine-tuning, LoRA, an MLP adapter — folds the document into weight deltas.

Each of these has its own literature and its own baselines, and the abstract's own framing is that there is no single winner among them. What this paper adds is a controlled comparison across five knowledge-intensive benchmarks, with the storage budget matched — which is what turns separate claims into one table.

The oracle setting is the easy case: the correct document is handed over and the method only has to answer from it. There the ordering is mild. Cartridges lead the parametric methods by 10 points, and Compaction keeps pace with Cartridges as long as the compression stays gentle — past 50x, Compaction falls behind even the parametric methods, by 10 points.

Then the paper moves to multi-document retrieval, where the method has to find the right document among many — the setting that actually resembles a retrieve-then-generate pipeline. Cartridges are the only method that keeps up with leaving the documents in context. An oracle benchmark asks only whether a method held the content; the multi-document benchmark also asks whether it can find it — and that second one is the scenario the paper calls the realistic one.

That is a different claim from "cartridges are better." One reading of the widening gap is that a KV state keeps a document locatable in a way a weight delta does not — but the paper reports the gap, not that explanation, so take it as a reading.

Take the one scenario the paper calls the realistic one — multi-document retrieval, where the method has to find the right document among many — and line up everything reported there. Cartridges match in-context learning: reading the documents directly is the reference the compressed methods are measured against, and a trained KV state matches it. Measured down from that same reference, Compaction sits 15 points below it and the parametric methods sit 29 points below. Subtract those and Compaction leads the parametric methods by 14 points — it gives up roughly half the distance from Cartridges down to the parametric baseline, while still storing knowledge in the very same place. The two KV methods are further from each other (15 points) than Compaction is from the parametric baseline (14), which is the clearest sign that "put it in the KV cache" is not one technique.

The oracle setting reports a much smaller gap for the same pair: Cartridges over parametric by 10 points rather than 29. Those two numbers come from different evaluation scenarios, so the abstract does not license subtracting one from the other into a price of retrieval. What the abstract does say is that multi-document retrieval is the more realistic of the two scenarios, which is why it is the one worth reading the ordering from.

The bill arrives on benchmarks the document has nothing to do with. Cartridges are the only KV method the paper finds suffering catastrophic forgetting — a 6% drop on control benchmarks, 13% on coding — which puts them in the same category as full fine-tuning and large MLP adapters rather than alongside Compaction. Alongside that retrieval advantage, Cartridges show a 6% degradation on control benchmarks and 13% in coding.

Compaction is not on that list, which is consistent with it compressing a cache rather than training one — though the paper reports the outcome, not that explanation, so treat the reasoning as a reading rather than a finding.

Which leaves a decision rather than a winner. If the lookup is hard and a general-capability regression is acceptable — a closed domain, a narrow product surface — the cartridge is the strongest option at the budget. If the base model has to stay intact, Compaction at mild compression buys most of the benefit without the forgetting. And if the document is small enough that the per-query KV memory is affordable, leaving it in context is still the thing everything else is trying to approximate.

Goes deeper in: LLM Internals → KV Cache → Memory Cost

Related explainers

• Parametric Memory Law links LoRA capacity to verbatim recall — the parametric side of this comparison, measured: how much a LoRA of a given rank can actually hold.
• Compress reasoning KV caches 5.8x with beacon queries — the compaction side, and what decides which KV entries survive the squeeze.