What are document-keyed evidence views?

A stored, per-word importance table for one retrieved document, accumulated from the generator's attention over every past retrieval of it. Because the table is keyed to the document rather than to a query, any later query can render a short plain-text view of that document from it at whatever token budget the request has.

Why does keying the scores to the document instead of the query matter?

It changes how often the expensive part runs. Query-keyed compression has to redo the work for every request, so a popular document is compressed over and over; document-keyed scores are paid once and amortised over every future retrieval. REVA reports 5.3–15.6× lower compression overhead on that basis.

Does a cheaper view mean a worse answer?

The paper reports the opposite on its four benchmarks — generation quality up 1.0–5.8 points. The plausible reason is that importance aggregated over many real accesses estimates what matters in a document better than a single query-conditioned pass. The case to watch is the unusual query against a popular document, where the accumulated view was shaped by readers who wanted something else.

How is this different from prefix caching?

They are the same bargain at different layers. Prefix caching reuses computed KV state for a shared token prefix; REVA reuses an importance judgement about a document's words to build the text that goes into the prompt in the first place. Both key the reuse to content rather than to a request, and both pay off in proportion to how repetitive the traffic is.

LLM· 2026-09-13

REVA cuts RAG compression overhead up to 15.6× — Document-keyed evidence views — What does it mean?

TL;DR

What is it: REVA (Reusable Evidence View Aggregation, arXiv 2609.11209) is a serving-side layer for retrieval-augmented generation that builds document-keyed evidence views — one reusable, pre-scored short version of each retrieved document, mined from the generator's own past attention rather than recompressed for every query.
Why it’s needed: Retrieved documents usually dominate the length of a RAG prompt, so anything you do to shorten them runs on every single request — and where retrieval traffic is skewed, that is mostly the same expensive work redone on the same few popular documents.
vs previous: Per-query context compression runs a fresh compressor pass for each query and throws the result away, so a frequently retrieved document is compressed again on every retrieval; REVA keys the importance scores to the document instead, so the work is paid once and reused across queries.

Jargon

RAG: Retrieval-Augmented Generation. The model answers from documents fetched at query time instead of from its weights alone. Primer
Context compression: Shortening retrieved text before it enters the prompt, so the model sees the parts that matter and you pay for fewer tokens.
Token budget: The cap on how many tokens the retrieved evidence may occupy in one request. It moves with the model, the price tier, and how much room the rest of the prompt needs.
Attention weights: Per-token scores the model computes over the prompt while answering — in effect, which words it leaned on. How the scores are computed
Word units: REVA's readable granularity. Token-level attention is folded up onto whole words, because a view has to come out as plain text the generator can read back.
Evidence view: The short, plain-text version of one document that REVA renders for a given token budget, in the document's original order.
Budget-agnostic store: The stored scores carry no budget of their own, so the same store renders a tight view or a generous one with no recomputation.
REVA: Reusable Evidence View Aggregation — the paper's name for the whole scheme.

The news. On September 10, 2026, a paper introducing REVA — Reusable Evidence View Aggregation — appeared on arXiv as 2609.11209. It compresses frequently retrieved documents once, from historical query–document–model interactions, rather than once per query. Across four benchmarks with current LLMs the authors report generation quality up by 1.0–5.8 points, compression overhead down 5.3–15.6×, and under 40 ms of added latency. Read the paper →

Picture the trail, not the compressor. A popular trail gets walked again and again, and every walker independently works out which bends are worth stopping at. Per-query context compression is the park hiring a surveyor to walk the whole trail again for each visitor, just to hand that one person a shortlist of stops. The ranger's alternative asks for no extra survey: visitors are already stopping where the views are, so mark the map from where they stood.

REVA does the ranger's version. Every time the generator answers using a retrieved document it computes attention weights over that document's tokens as part of ordinary decoding — the signal is a by-product of answering, not a separate scoring pass over the document. REVA folds those weights up onto word units, so the signal is about readable words rather than subword fragments, and adds them into a running per-word score for that document. The scores are keyed to the document, not to the query that happened to fetch it — which is the whole reason the work amortises. Every repeat retrieval of the same document adds another round of evidence about which of its words carry answers.

At serve time there is no compressor to run. Given a token budget, REVA renders a plain-text view sized to that budget from the stored scores — in practice keeping what those accumulated scores rank highest — and emits it in the document's original order. The store is budget-agnostic — the same scores render a tight view or a generous one — and because the output is ordinary text handed over the normal RAG interface, nothing downstream needs to know it happened. If the shape feels familiar, it is the trade prefix caching makes one layer down: pay once for work that many later requests share, and key it to the content rather than to the request.

Retrievetop-k documents for this query

Read the storescores keyed by document, not query

Render to budgetplain text, sized to budget, original order

Generatethe answer the user asked for

Fold attention backthis run's attention updates the store

Approach	What runs per query	What gets reused	Reported effect
Per-query context compression	a full compressor pass over the retrieved text	nothing — the shortened text is discarded	the baseline REVA is measured against
REVA, document already in the store	a plain-text render from the stored scores	per-word scores for that document, from every past retrieval	5.3–15.6× lower compression overhead, +1.0–5.8 quality points (paper)
REVA, document seen for the first time	the ordinary path — there is nothing stored yet	nothing on that first retrieval	the gain arrives with repeat retrievals, not the first one

Hold one document fixed and count the compressor passes. Say a policy PDF is the document that answers most questions in your corpus, and it is retrieved by 40 queries in a day — an illustrative number, not the paper's setup — and one compressor pass over it costs 300 ms of request overhead (also illustrative). The per-query path pays that 300 ms on every retrieval: 40 × 300 ms = 12,000 ms of compressor work, for one document, in one day. REVA runs no compressor pass at all, because the scores came from attention the generator had already computed, so each query pays only the render. The paper reports under 40 ms of added latency; take 40 ms as a conservative ceiling, so the render side costs at most 40 × 40 ms = 1,600 ms. Because that is a ceiling, the ratio it gives is a floor: twelve seconds of compressor work against at most 1.6 seconds of rendering is a reduction of at least 7.5× — a number that lands inside the 5.3–15.6× band the authors measure rather than deriving it. That band is wide for a reason worth internalising: the multiple depends on both ends of the ratio — mostly on how expensive the compressor you replaced was, and partly on what the render itself costs.

Where the idea earns its keep, and where it does not. The store is built from historical interactions, so the advantage is concentrated on documents retrieved again and again; a document seen for the first time has no accumulated view and nothing to reuse, which is why the third row of the table above exists. How mild that constraint is depends on your corpus. Where retrieval traffic is skewed — a small set of documents answering a large share of questions — it barely bites, and it is the same skew that makes caching pay off at all. Where every query reaches a different document, there is little for the store to amortise over.

The sharper question is semantic rather than economic. A document-keyed view is, by construction, not tailored to your query: asking it about an unusual corner of a popular document means asking for words most previous readers had no reason to dwell on. The reported quality gain of 1.0–5.8 points suggests that aggregating across many real accesses is, on average, a better importance estimate than one query-conditioned guess — but on average is carrying weight in that sentence, and the failure mode to watch is the rare query against the well-trodden document. Measure it as a tail-latency and tail-quality question, not as a mean.

Goes deeper in: AI Agents → Retrieval & RAG → Retrieve-Then-Generate

Related explainers

One compressor, many budgets (FlexComp) — the other way to serve any token budget: make the compressor ratio-adaptive instead of storing scores.
Reordering RAG evidence for prefix-cache reuse (CacheWeaver) — the neighbouring trick: leave the evidence alone and change its order so the cache hits.
Prompt compression by a learned encoder (Latent Context LMs) — a trained model doing the shortening, for contrast with mining the generator's own attention.
Entropy-aware KV-cache compression (InfoKV) — the same keep-or-drop question one layer down, inside the KV cache.

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based