What is CliffCompaction?

A context-compaction method for coding agents from Carnegie Mellon and Bosch. When the context crosses a token threshold, it drops long tool results, cuts tool calls to signatures, truncates thoughts, and **discards the previous compaction** instead of folding it into the new one. It never rewrites text.

Why is a summary of a summary a problem?

Each summary can lose or distort details, and the next summary is built from that already-lossy text, so errors compound. The paper also finds that a plausible summary **discourages the agent from re-checking source files**, so an omission is less likely to be recovered.

How does CliffCompaction relate to prompt caching?

It leaves the context untouched between compactions, so the provider's cached prefix stays valid on every turn. Only each compaction forces a re-prefill, and uncached input costs roughly **5 to 6 times** cached input for the models in the paper.

What does CliffCompaction give up?

Direct long-history recall. After two compactions, the direct record of early turns is gone, although its influence may survive in later actions or code. The authors call this trading **compaction recall** for **compaction precision**.

Agent· 2026-09-24

CliffCompaction — Re-compacting from the original context — What does it mean?

TL;DR

What is it: The CliffCompaction paper (arXiv 2609.26779, September 22, 2026) is a context-compaction rule for coding agents: when the context fills up, shrink only the turns since the last compaction, keep what survives word for word, and throw the previous compaction away.
Why it’s needed: Long agent sessions eventually exceed a fixed context window, so a harness must manage old context, through compaction or another memory strategy, and that choice decides what the agent forgets and how often the provider must recompute the context at the uncached price.
vs previous: The common alternative is LLM summarization that folds each old summary into the next one, so every compaction is a summary of a summary and small errors pile up (context drift); CliffCompaction never rewrites text and never compacts a compaction.

Jargon

Context window: The maximum number of tokens a model can read in one request. An agent re-sends its whole conversation every turn, so a long session eventually hits this limit.
Compaction: Shrinking an agent's conversation history so it fits the context budget again. It can truncate, drop, or summarize old turns.
Context drift: The slow loss of accuracy when a compacted history leaves out or distorts something, and the agent keeps working as if it still had the full picture.
Compaction precision: The paper's term for how faithful the retained history is: nothing kept is distorted. Verbatim excerpts score high; rewritten summaries score lower.
Compaction recall: The paper's term for how much of the older history is still available after many compactions. Chained summaries score high; CliffCompaction deliberately scores low.
Cached input tokens: Prompt tokens the provider already holds as KV cache from an earlier request with the same prefix. They are billed far below fresh input. Prompt caching covers the pricing.
Re-prefill: Recomputing the KV cache for a context from the first changed token onward. Any edit to old context forces one, billed at the uncached price.
Tool-call signature: A tool call reduced to its name and essential arguments, such as the file path, with long inline content removed. The agent can re-run it to get the content back.

The news. On September 22, 2026, researchers at Carnegie Mellon University and the Bosch Center for AI posted CliffCompaction, an automatic compaction method for long-horizon coding agents. They report up to 50% lower cost under a bounded context while matching or improving success on Terminal-Bench 2.0 (a benchmark of command-line tasks), and CUDA-kernel speedups on KernelBench (a benchmark where the agent writes faster GPU kernels) of 2.23× after 200 steps and 3.58× after 400 steps. They also release an API proxy that works with any scaffold (the agent harness that runs the loop), so it runs in front of Claude Code, Codex and other harnesses. Read the paper →

Why copies of copies go grey

Picture the left half of the illustration. You photocopy a page, then photocopy the copy, then photocopy that. Each copy looks almost as good as the one before it, so nobody stops to check. After a few rounds the fine print is gone, and it went missing a little at a time, so no single copy looked broken.

That is what chained summarization does to an agent's memory. The harness fills the context window, asks a model to summarize the older turns, and keeps going. When the window fills again, the new summary is written from the old summary plus the new turns. The paper names the specific harm: a plausible summary makes the agent less likely to re-read the files and docs where an omitted detail still lives. A missing detail does not produce an error; it produces a confident agent working from a wrong picture. That is one of the four context failure modes in a new form.

The right half of the illustration is the fix. Always copy from the original, and shred the old copy. When the context crosses its token threshold, CliffCompaction compacts only the live session since the last compaction and discards the previous compacted block completely. Every compaction is therefore exactly one step away from the real turns, never two or five.

Session growscontext left untouched

Threshold hitwindow budget reached

Discard old compactionnever nested

Compact live turnsdrop and truncate only

Grow againcache valid until next cliff

What gets kept, word for word

Compacting "only by dropping and truncating" needs a rule for what to drop. The authors measured where the tokens go: on Terminal-Bench 2.0 with GLM 5.1 and no compaction, tool results were 56% of the context and tool calls another 28%. So the rules target tool traffic:

Tool results longer than 500 characters are dropped. Short ones, such as grep matches and exit codes, are cheap and usually useful, so they stay.
Tool calls shrink to a signature: the tool name, the target path and the essential arguments. A file-write call loses its inline file body, because the file still exists on disk.
Agent thoughts are truncated to 300 characters. The system prompt and the task description are kept in full, and the most recent turns are kept unchanged.

The signature is the sticky note in the illustration. Dropped tool results can usually be recovered by re-running the preserved call signature. A dropped 4,000-character file read, for example, comes back with one more read of the same path. Truncated thoughts and discarded compactions cannot be fetched back this way. Notice what is absent from the list: no model call and no rewritten sentence. The compactor is a deterministic filter, so it adds no LLM cost and cannot put words in the agent's mouth.

The cliff is also a cache decision

The name comes from the shape of the context-length curve: it grows smoothly, then drops sharply at each compaction and grows again. That shape is deliberate, and the reason is prompt caching. A provider stores the attention keys and values it computed for a prompt (the KV cache) and can reuse them only for the part of the prompt that is byte-identical to the previous request, the same prefix match a serving engine does. Any edit to old context forces a re-prefill from the edit point onward, and the paper reports uncached input costing roughly 5 to 6 times as much as cached input for the models it studies.

A sliding window, which drops the oldest turn whenever a new one arrives, edits the context on every turn once it is full. CliffCompaction edits it only at the cliffs, so the cache stays valid across each whole growth segment.

Worked example (illustrative). Hold three things fixed: a 128K-token context budget, uncached input priced at 5× cached input (the low end of the paper's range), and output tokens ignored. Call one cached token 1 unit. A sliding window that is full re-prefills almost the whole window every turn: about 128K × 5 = 640K units per turn. CliffCompaction reads at most 128K cached tokens per turn, 128K units. Suppose each compaction shrinks the context to 20K tokens and fires every 50 turns: that is 20K × 5 = 100K units once, or 2K units per turn spread over the segment. The total is ~130K vs 640K units, roughly 5× less input cost per turn in this setup. The paper's measured gap is smaller but in the same direction: on KernelBench Level 3 with Kimi K2.7 over 400 steps, the sliding window cost $21.52 and CliffCompaction $8.32, about 2.6× less.

Method	What happens to old context	Rewrites text?	KernelBench L3 speedup · cost
Sliding window	Oldest turns dropped; system and task prefix kept	No	2.86× · $21.52
Summarization	Older turns rewritten into an LLM summary	Yes	3.47× · $8.10
Summarization + microcompaction	Old tool outputs cleared first; summarized when that is not enough	Yes	3.33× · $12.84
CliffCompaction	Long tool results dropped, calls cut to signatures, previous compaction discarded	No	3.58× · $8.32

The price: it forgets on purpose

The table row for summarization is a reminder that this is a tradeoff, not a free win. Summarization was slightly cheaper on KernelBench, and on the short SWE-bench Verified tasks every method in the comparison landed within about 2.6 points. CliffCompaction buys faithfulness (precision) by giving up direct long-history recall: after two compactions, the direct record of an early session is gone from the context, whereas a chain of summaries might still carry a blurred version of it.

The authors argue the loss is softer than it looks, through what they call residual propagation. Every turn in the current session was written while the previous compaction was still visible, so the agent's own actions and code carry forward what it had learned, even after the text that taught it is gone. The results where it matters most are on the longer tasks: on Terminal-Bench 2.0 at a 16K budget, swapping the scaffold's built-in LLM summarizer for CliffCompaction raised Kimi K2.6 from 55.45% to 61.42%, and GLM 5.1 went from 49.83% at full context to 54.33% at 16K while its cost per task fell from $0.54 to $0.27.

Two limits from the paper itself. The benefit shows up only on medium-to-long tasks, and the minimum usable budget depends on how much of it the scaffold's system prompt and tool definitions already consume. And the comparison covers only methods that edit the conversation at inference time: it does not test agents trained to manage their own context, or external memory stores.

Goes deeper in: AI Agents → Context Engineering → The 4 Fixes

Related explainers

Turn amplification in agent context compression — the cost trap on the other side: compressing tool output can make an agent take more turns, each re-billing the whole context.
Prefix Sliding — a related bounded-memory rule for long reasoning chains that pins the prompt prefix and drops the middle.

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based