The news. On May 14, 2026, the paper "Does RAG know when retrieval is wrong?" was posted to arXiv. The authors introduce Context-Driven Decomposition (CDD) — a diagnostic prompt pattern that breaks a RAG query into a retrieval claim, a parametric claim, and an explicit conflict-resolution step. Headline numbers: standard RAG hits 15.0% under misconception injection; CDD reaches 71.3% on temporal-shift cases; Gemini lands at 64.1% with CDD on the same set, with Claude variants improving unevenly. Read the paper →

Picture the courtroom. The judge needs to rule on a question of fact. There are two witnesses available — one is the document (a retrieved passage, possibly with a planted false detail), the other is the expert (the model's own training-set memory of the same topic). Standard RAG is what happens when you let one assistant summarize both testimonies into a single brief and the judge rules from that brief. If the document and the expert disagree, the disagreement vanishes into the summary — and the ruling is whatever the summary happened to emphasize.

CDD is what happens when you call each witness separately, get each one's statement on the record, and then explicitly ask: "Where do they agree? Where do they contradict? Which one wins, and why?" — before the judge is allowed to rule. Same evidence, same question, dramatically different verdicts.

This is what the authors are after. The same RAG stack — same retriever, same model, same context — produces a wrong answer when given a fused prompt and a correct answer when given the decomposed sub-prompts. The mechanism that changes is the prompt, not the model. CDD is a prompt-level intervention that any retrieval & RAG pipeline can adopt without retraining, without re-indexing, without touching the embedding store.

What "misconception injection" actually tests

The paper's adversarial setup is simple: take a question that has a known correct answer, retrieve a passage that contains the right information, and then slip a plausible-sounding false claim into that retrieved passage. The model now sees context that mostly agrees with reality but is locally wrong on the specific fact under test.

Standard RAG reaches 15.0% accuracy under this attack. That is not "below human" or "worse than the no-context baseline" — it is catastrophically wrong. The model is effectively trusting the planted misconception over its own pre-training. This is the failure mode that single-pass RAG cannot expose to itself: the prompt fuses Q + context, the model answers, and there is no internal step where "but wait, the document says 2024 and I learned 2019 — which do I trust?" can surface.

CDD changes the dynamic by forcing that question to be asked out loud. The conflict-resolution sub-prompt cannot be skipped — it is a separate model call, with its own input (the two extracted claims) and its own output (a comparison verdict). The model has to commit to a position on whether the document overrides memory, on this specific contradiction, in writing, before it produces an answer.

Where the numbers come from

The headline gap — 15.0% → 71.3% — is on temporal-shift cases, where the world moved on after the model's training cutoff. Here is the structure that produces it:

The transfer across model families is the second-order claim worth marking. CDD is not a fine-tune — it is a prompt pattern. The fact that the same three sub-prompts produce different magnitudes on Gemini vs Claude vs the paper's primary model suggests that context-compliance is partly a model-level property (how willing the model is to override its own memory) and partly a prompt-level property (how forced the override is).

A worked example makes the gap concrete. Suppose the question is "When did X happen?" The model's training data says 2019. The retrieved context says 2024 (correct, current). Under misconception injection, the same retrieved context might also include a slipped sentence "Note: earlier reports of 2024 were preliminary; the corrected date is 2017" — a plausible-sounding contradiction.

Standard RAG sees the fused prompt, weighs the conflicting signals implicitly, and often answers 2017 or 2019 — both wrong — because the model has no explicit step to surface what it just read vs what it already knew. CDD's first sub-prompt extracts the retrieval claim ("Context says: 2024, possibly corrected to 2017"), the second extracts the parametric claim ("My pre-training memory says: 2019"), and the third sub-prompt is forced to rule on the contradiction with both claims on the table — at which point the model can notice the "correction" is suspiciously inconsistent with everything else in the context and that the original 2024 claim agrees with reality. The result on temporal-shift evaluation is 71.3% accuracy — a roughly 4.7× swing over standard RAG's ~15% under injection. (Numbers from the paper; the worked example is illustrative of the mechanism, not a quote from the dataset.)

This is also why CDD pairs naturally with production drift detection. Once you have a measurable "context-compliance" axis distinct from retrieval and generation, you can monitor it in production — and a sudden drop on temporal-shift cases is the early signal that the index is stale or the model's parametric memory has drifted relative to the corpus. Without CDD-style decomposition, that signal hides inside the aggregate answer-accuracy number.

Why the prompt change matters more than it sounds

The CDD finding sharpens a debate that has been running through RAG evals for two years: is the RAG system failing at retrieval, at generation, or at the bridge between them? Most eval suites measure retrieval (recall@k, MRR) and generation (answer accuracy) but treat the bridge as a black box. When the answer is wrong, you can rarely tell whether the retriever pulled the wrong chunk, the model ignored the right chunk, or the model trusted the wrong fact in a chunk that was otherwise correct.

CDD isolates the third case. The decomposition makes the model write down what it would have said from each side before committing to an answer — turning "did the model use the context correctly?" from a guess into something you can grade directly. This is the same pattern that agent failure-mode analysis is moving toward across the board: replace the implicit fused-prompt step with an explicit, separately-graded sub-step, and the failure becomes locatable instead of just measurable.

Goes deeper in: AI Agents → Retrieval & RAG → RAG Failure Modes

Related explainers

• Is Grep All You Need? — Grep vs vector retrieval for agentic search — pairs naturally; CDD assumes retrieval already happened and isolates what the model does with conflicting context
• FutureSim — Harness-level agent eval vs single-shot QA — also pushes evals beyond single-shot, surfacing failure modes a one-shot prompt can't expose
• MCP SEP-2663 — async task handles — the protocol counterpart for breaking one "tool call" into a multi-step exchange