MemOps is a benchmark that evaluates an AI agent's long-term memory as a sequence of lifecycle operations — remember, forget, update, and reflect — instead of treating memory as a static store of facts to look up. It embeds those operations inside long, task-oriented conversations and records a structured trace of each memory event (its trigger, target, scope, state transition, and supporting evidence), so a failure can be attributed to the specific operation that broke rather than only marking the final answer right or wrong.

Why does grading memory operations matter?

Because a single right/wrong verdict on the final answer hides which of several very different bugs occurred. A missed write, a fact that should have been forgotten but lingered as stale state, an update that landed on the wrong target, and a bad reflection over correct facts can all surface as the same wrong answer, yet each needs a different fix. Tracing operations tells you where memory broke, which is the only part you can act on.

How does MemOps relate to normal agent memory like RAG?

Retrieval-augmented generation and many memory tests typically focus on reading the right fact back out of a store. MemOps widens the lens to the full lifecycle: it also grades whether the agent wrote the fact in the first place, retired it when it expired, and rebound it correctly when it changed. In that sense it is closer to observability — a structured trace of what memory did over time — than to a retrieval accuracy score.

MemOps benchmarks agent memory as lifecycle operations — Memory lifecycle operations

Jargon

MemOps: The benchmark under discussion. It embeds memory operations inside long, task-oriented conversations and grades the operations, not just whether the final answer was right. It is a diagnostic benchmark, not a serving optimization — it tells you what broke, it does not make memory faster.
Memory lifecycle operations: The four things an agent's memory actually does over time: remember (write a new fact), forget (retire a fact that no longer holds), update (change a fact that changed), and reflect (reason across stored facts). MemOps also tests compositions of these.
Structured memory trace: The record MemOps keeps for each memory event: its trigger (what in the conversation caused it), its target (which fact it touched), its scope, the state transition it made, and the supporting evidence for it. The trace is what lets a wrong answer be traced back to one operation.
Forget: Here it means superseding a fact that is no longer true (you moved apartments), not deleting a row from a database. Forgetting too late leaves the agent citing stale state; forgetting too eagerly drops something you still needed.
Update: Rebinding a changed value to the right fact. The failure the benchmark names here is a wrong-target update: the new detail gets attached to the wrong fact, so the agent is confidently wrong about something it "knows".
Reflect: Reasoning across several stored facts to answer a question, rather than reading back one fact. A reflection can go wrong even when every underlying fact is stored correctly, because the mistake is in the reasoning across them, not in any single fact.
Static fact store: The mental model MemOps argues against: memory as a bucket of facts you retrieve from. It hides when and how a fact entered, changed, or expired — exactly the timeline where long-horizon conversations go wrong.

The news. On July 14, 2026, a group released MemOps, a benchmark that reframes long-term agent memory as a sequence of lifecycle operations rather than a static fact store. It embeds memory operations into long task-oriented conversations and records a structured trace for each event — trigger, target, scope, state transition, and supporting evidence — so evaluation can separate remembering, forgetting, updating, reflecting, and their compositions. The stated goal is diagnostic: to tell whether a model missed a fact, updated the wrong target, or used stale state, instead of only marking the final answer right or wrong. Read the paper →

Picture a personal assistant keeping one running notebook about you. Over weeks, they write a new line when you mention something ("she's allergic to shellfish"), cross out a line that stops being true ("moved out of the Oak St apartment"), rewrite a line whose detail changed ("new address: 12 Pine"), and, when you ask a question, read across all the lines to answer it. When the assistant sends your package to the old apartment, the mistake is not that "the notebook is bad." It is that one of those four bookkeeping steps slipped — and until you know which one, you cannot fix it.

That is the whole idea MemOps makes measurable. Many memory benchmarks hand the agent a long conversation and then check the final answer: right or wrong. But a final answer is a single bit laid over many memory events, and it compounds every earlier error into one verdict. A wrong answer tells you the memory failed; it does not tell you where, and "where" is the only thing you can act on. MemOps instead does error analysis first: it keeps the structured trace, so the failure resolves into a named operation.

Those operations are not interchangeable — each is a different way for memory to go wrong, and each wants a different fix.

Operation	What it does	How it fails	What a final-answer score hides
Remember	write a new fact the first time it appears	the fact never gets written	the agent looks forgetful when it simply never encoded the detail
Forget	retire a fact that no longer holds	an expired fact lingers as stale state	the agent cites something that used to be true
Update	rebind a changed value to the right fact	the new value lands on the wrong target	the agent is confidently wrong about a fact it "knows"
Reflect	reason across several stored facts	a wrong conclusion drawn across facts that are each correct	every underlying fact was right, yet the conclusion was wrong

Read the last column top to bottom and the point lands: four completely different bugs can all surface as the same wrong final answer. A pass/fail score puts them in one bucket labelled "memory: broken." The trace splits the bucket back apart. This is the pass/fail-versus-score problem, aimed squarely at memory — and it is why MemOps looks less like a leaderboard and more like the span-per-tick traces you would run in production to see what an agent actually did.

Here is why the distinction pays off, with illustrative arithmetic. Say a 30-turn conversation produces 12 memory events, and the agent gets the final question wrong. A final-answer benchmark records 1 question, 1 wrong — memory: 0%, and you are left to guess. Trace the events and suppose 11 of the 12 were handled correctly; the one that failed was an update at turn 14 that rebound your new address to the wrong contact. Now the picture inverts: the memory system was right on 11 of 12 operations (~92%), and the entire failure is one wrong-target update. The fix is not "rebuild memory" — it is "harden the update path." The single-bit score threw away the 11 correct operations and mislabelled a one-line bug as a total collapse. (Numbers here are illustrative — the paper reports the tracing method, not these counts.)

None of this makes memory better on its own — MemOps is a ruler, not a repair. What it buys is knowing what to log and what to watch: once a failure is attributed to forget-too-late versus wrong-target update, the failing operation is the thing you instrument, alert on, and regression-test. For anyone wiring up an agent that has to remember across sessions, that is the transferable lesson — before you trust a memory score, ask which operation it is actually grading, because "remembered it" and "read it back right" are not the same skill.

Goes deeper in: AI Agents → Evals & Diagnostics → Error Analysis First

Related explainers

AtomMem — Atomic-fact memory — a complementary idea: storing memory as small, checkable facts instead of raw transcript gives each operation a clean unit to act on
EvoMem — Patch-based agent memory — the update operation as a first-class mechanism: apply diffs to memory rather than overwrite it
AgenticSTS — The memory contract — giving memory operations a contract so they can be validated, not just performed
Proactive memory — Reminder injection — the reflect/retrieve side: deciding when a stored fact should resurface on its own

Frequently Asked Questions

Check what you knowMap your AI & GPU knowledge across every track — free, role-based