What is counterfactual memory auditing?

A check run before a preference is written into an agent's long-term memory. The system removes the candidate write, recomputes what it believes about the user without it, and keeps the write only if evidence other than the write itself still supports it. A claim that has nothing but itself behind it is treated as suspicious.

How is preference drift different from memory poisoning?

Preference drift is the user genuinely changing — they really do prefer aisle seats now. Memory poisoning is a false preference planted by untrusted text the agent read. At the moment of the write the two are indistinguishable, which is why CAPTURE judges them by the surrounding evidence rather than by the claim itself.

Does CAPTURE stop memory poisoning?

No, it reduces it. The paper reports poisoning still succeeding in 11.5% of fixed attacks and 24.7% of adaptive ones, where an adaptive attacker plants corroborating evidence so the audit passes. It is one filter in a layered defense, not a wall.

What does it cost to run the audit?

Two things. Compute is cheap, because the belief is already a probability distribution and dropping one piece of evidence to re-score is a calculation the system performs anyway. The real cost is recall: the paper accepts 83.5% of genuine updates, so roughly one real change of mind in six is rejected and has to be repeated.

How does this relate to context engineering?

Memory poisoning is one of the four context failure modes taught in the Context Engineering module. Counterfactual auditing is a concrete defense against that specific mode, applied at the moment a write is proposed rather than after a poisoned fact is already steering behaviour.

Agent· 2026-09-06

CAPTURE separates real preference change from poisoned memory — Counterfactual memory auditing — What does it mean?

TL;DR

What is it: A new paper, CAPTURE (arXiv 2609.02265), replaces unconditional memory accumulation with counterfactual memory auditing — a check that asks whether anything other than a new write itself stands behind storing it.
Why it’s needed: An agent that remembers you has to accept that you changed your mind while rejecting a preference planted by whatever text it just read, and at the moment of the write those two look identical — CAPTURE reports 83.5% of genuine updates accepted while adaptive poisoning still lands 24.7% of the time.
vs previous: The previous approach was unconditional accumulation — append every stated preference and trust it — which cannot separate preference drift from memory poisoning, because it never asks what evidence besides the claim stands behind the claim.

Jargon

Preference drift: Your tastes genuinely changing over time — you really did start preferring aisle seats. CAPTURE models it as a latent state that keeps moving, including between the moments you talk to the agent, rather than as a one-off correction.
Memory poisoning: A false preference written into an agent's long-term memory by untrusted content the agent read, so the agent keeps acting on it long after the attacking text is gone. It is one of the four context failure modes in Context Engineering.
POMDP: Partially Observable Markov Decision Process. A model for deciding when you cannot observe the true state — here, what you actually prefer — only noisy evidence of it. Continuous-time means the state is allowed to drift in the gaps between interactions, not only during them.
Belief state: A probability distribution over what your preferences currently are — the latent state the paper says it is inferring — held instead of one stored answer. A new claim updates the distribution rather than overwriting a fact, which is what makes it removable later.
Neural ODE: A neural network that outputs a value's rate of change rather than its next value, so the value can be rolled forward by any amount of elapsed time. CAPTURE uses one to age the belief state forward to now before judging a new claim.
Multi-timescale ledger: Two records kept side by side: transient evidence (this session, easily revised) and durable evidence (repeated across sessions). A claim supported only by transient evidence is exactly the one worth auditing.
Fixed vs adaptive attack: A fixed attacker plants the same text whatever the defense does. An adaptive attacker knows the audit exists and plants corroborating evidence to survive it. In this paper adaptive attacks succeed more than twice as often.

The news. On September 2, 2026, researchers released CAPTURE, a personalization method that treats a user's preferences as an evolving latent state to be inferred rather than a list of facts to accumulate. Across 480 episodes from 96 users it reports a 71.5% preference win rate against baselines at 69.3% and 66.1%, accepts 83.5% of genuine preference updates, and holds poisoning success to 11.5% for fixed attacks and 24.7% for adaptive ones. The claim underneath the numbers is that a memory write should have to earn its place rather than merely be stated. Read the paper →

Your bank keeps a statement for you, one line per charge, and this morning a new line appears from a city you have never shopped in. There are exactly two stories behind it. You moved, and the charge is yours. Or your card was cloned, and it is not. The line itself cannot tell the bank which story is true — it reads identically either way. So the analyst does the one thing the line cannot do for itself: they cover it up and read the rest of the statement. Rent paid in that same city for two months, a flight booked three weeks ago, a run of local coffee shops — and the charge stops looking odd. Nothing behind it but the charge itself, and it stays odd.

An agent's memory of you has the same shape, one line at a time. It reads "I prefer aisle seats" and has to decide whether to write that down. Maybe you said it. Maybe it appeared in a support-ticket thread, a web page, or a shared document the agent was asked to summarize — the ordinary way untrusted text reaches a model that also holds your data (the lethal trifecta). Once written, a preference behaves like any other remembered fact: it survives the conversation that carried it, and it steers behaviour with no surviving trace of where it came from. That is memory poisoning, and appending first and asking later does not work, because by then the poisoned line is indistinguishable from a real one.

Counterfactual memory auditing is the analyst's move applied to a memory write: drop the candidate write, recompute what the system believes about you without it, and keep it only if the remaining evidence still points the same way. The word counterfactual is doing real work — the system is not scoring the claim, it is scoring the world in which the claim was never made. A genuine change of mind usually leaves a trail: you booked an aisle seat last month, you complained about the window twice. A planted claim usually has nothing behind it but itself.

New claima preference the agent just read

Age the beliefroll the state forward to now

Drop and re-checkdoes other evidence still support it?

Commit, reject, askand ask when the answer is close

In the paper's own terms, the belief about you is a continuous-time POMDP, and a neural ODE carries that belief forward through the hours or weeks since you last spoke, so a claim is judged against who you are estimated to be now, not who you were at the last write. Evidence lands in a multi-timescale ledger that keeps transient signals apart from durable ones — the two bands on the statement. The audit is affordable precisely because the belief is already probabilistic: removing one piece of evidence and re-scoring is a calculation the system was set up to do anyway, not a second model that has to be trained and served.

The fourth piece is what happens when the audit comes back close. Rather than guessing, uncertainty triggers a clarifying question — the "was this you?" text — which is the same discipline as an agent that pauses to observe instead of committing on thin evidence. It costs the user a turn, and it is the only branch that is allowed to be wrong cheaply.

The audit is a filter, not a wall: an attacker who knows it is there plants the corroborating evidence first, which is why adaptive attacks land more than twice as often as fixed ones here. That is the honest reading of the result, and it is also why this belongs alongside the other input filters in a layered defense rather than replacing them.

What the paper measures	Reported	How to read it
Preference win rate	71.5%	against two baselines reported at 69.3% and 66.1% (CAPTURE)
Genuine preference updates accepted	83.5%	real drift the audit lets through; the other roughly 1 in 6 is a real change it rejects (CAPTURE)
Fixed poisoning attacks that succeed	11.5%	attacker does not adapt to the audit (CAPTURE)
Adaptive poisoning attacks that succeed	24.7%	attacker shapes evidence to pass the audit; more than double the fixed rate (CAPTURE)
Evaluation scale	480 episodes, 96 users	5 episodes per user on average (CAPTURE)

Put the two rates against each other on a fixed amount of traffic. Say an assistant sees 1,000 candidate memory writes in a month, and that 90% of them are genuine and 10% are adaptive poisoning attempts (illustrative — the paper reports per-attack success rates, not a base rate of attacks in the wild). Accept everything, the old way, and all 900 genuine updates land — but so do all 100 poisoned ones, so 100 / 1,000 = 10% of the ledger is poisoned. Run the same traffic through the audit at the paper's rates: 900 × 0.835 = 752 genuine writes accepted, and 100 × 0.247 = 25 poisoned writes still get through, for 777 accepted lines of which 25 / 777 ≈ 3.2% are poisoned. CAPTURE cuts the poisoned share of the ledger from 10% to about 3.2%, and pays for it by dropping 148 real preference changes — roughly one in six times you change your mind, the agent keeps the old answer until you say it again. Which side of that trade you want is a product decision, not a modelling one: a travel assistant that forgets you now prefer aisle seats is annoying, and one that has been taught you prefer a hotel you have never stayed in is worse.

Goes deeper in: AI Agents → Context Engineering → The 4 Failure Modes and Agent Engineering → Layered Guardrails → Input Filters

Related explainers

GateMem — the memory governance trilemma — asks who is allowed to read and delete a shared memory; CAPTURE asks whether a write should have been made at all
MemOps — memory lifecycle operations — names the operations (remember, forget, update, reflect) that a counterfactual audit sits in front of
EvoMal — skill-library self-poisoning — the same attack aimed at the skills an agent writes rather than the preferences it remembers

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based