RIKA admits agent memories only on evidence — Evidence-based memory admission — What does it mean?
The news. On October 15, 2026, researchers at UC Davis and the University of Illinois Urbana-Champaign posted Reliable Knowledge Acquisition for Continual Learning Agents (RIKA). Auditing four existing memory systems, they found roughly 12–33% of the individual claims each had stored made later tasks worse when reused. RIKA pairs a four-type knowledge extractor with Evidence-based Knowledge Admission, and across six task streams and four model backbones it scored highest in aggregate on every backbone. Read the paper →
Picture a regional birdwatchers' club that prints a field guide every season, assembled from members' logbooks. Most members write down what they saw. Some also write down what they concluded: "this warbler lives in every valley." A guide that prints both kinds of line with the same confidence will eventually send a hiker to the wrong valley. The paper measures how often agent memory does the equivalent. It removes one stored claim at a time, reruns the later tasks, and calls a claim harmful when its effect on the score is reliably below zero. Across four systems and three task streams the harmful share ran from about 11.7% (Mem0, one stream) to 32.7% (Trace2Skill, one stream), and the authors trace the failures to claims that relied on information unavailable at reuse time, mismatched what later tasks needed, or went stale when the environment changed. A claim that sounds plausible is not yet a claim that is reliable.
RIKA's extractor first writes down four kinds of knowledge: factual, conceptual, procedural and metacognitive. EKA then treats every resulting claim like a field-guide entry and asks one question: does the feedback the environment already returned establish this claim, within the scope the claim states? An atomizer splits bundled notes into single claims. A routing agent then checks each claim against the feedback spans it cites. Claims the feedback states directly are directly grounded and go to the immediate route; claims that follow only by inference or abstraction are indirectly grounded and go to the deferred route; claims with neither are excluded. The route depends on scope, not on the kind of knowledge: a factual claim like "every schema version uses completed as the finalized label" is still deferred, because one observation of version 1 does not cover other versions.
| Claim (the paper's stylized database example) | Route | Why | Source |
|---|---|---|---|
In schema version 1, order amounts are stored in orders.amount | Immediate | The schema inspection output states it directly | Appendix C.2 and Section 3.2 |
| For realized-revenue queries, pending orders do not count | Deferred | Inferred from the records of one task, so its scope reaches past what was observed | Appendix C.2 and Section 3.2 |
All database versions use completed as the finalized label | Deferred | Factual in form, but a version 1 observation does not cover other versions | Appendix C.2 and Section 3.2 |
| Adding an index makes the query faster | Excluded | The trajectories hold no timing or query-plan feedback to derive it from | Appendix C.2 and Section 3.2 |
What keeps a deferred claim honest is a running tally. Each new claim is compared with the stored ones and labeled equivalent, specialize, generalize, contradict, overlap or independent. A later trajectory that grounds an equivalent or more specific claim adds 1 to the stored claim's evidence count e; one that grounds a contradicting claim subtracts 1; overlap and independence transfer nothing. Support flows from specific claims up to the general ones they entail, never the reverse. After every batch of tasks, a claim is admitted when e is at least its threshold: 1 for immediate claims and k for deferred ones, and this is re-checked for claims admitted earlier too. A claim stuck below its threshold for m consecutive batches is forgotten. The notes file the agent reads is then rebuilt from the admitted state each batch instead of being edited in place, because the authors cite findings that repeated memory rewrites can degrade stored knowledge.
| Claim in the paper's walkthrough (k = 3) | After batch 1 | After batch 2 | After batch 3 | In the notes file? | Source |
|---|---|---|---|---|---|
| Revenue rule (deferred) | 1 | 2 | 3 | Yes, admitted after batch 3 | Appendix C.2 |
| Overbroad fixed-label claim (deferred) | 1 | 2 | 1, after the schema change contradicts it | No, stays pending | Appendix C.2 |
| Metacognitive rule: re-inspect the schema after a failure (deferred) | not yet created | not yet created | 1 | No, stays pending | Appendix C.2 |
| Version-specific schema facts (immediate) | admitted at once | admitted | admitted | Yes | Appendix C.2 |
Walk the paper's own three-batch example, which is stylized and uses k = 3. In batch 1 a trajectory inspects schema version 1 and sees that filtering on status = completed gives the right revenue; batch 2 repeats the contrast on other records; after a migration, batch 3 hits a failing old query and finds the version 2 fields state and value. The revenue rule is re-derived from each of the three trajectories, so its count goes 1, 2, 3 and it is admitted exactly when it reaches k = 3. The overbroad "completed everywhere" claim climbs to 2, then the version 2 label settled counts against it and it drops to 1, so it stays out. The rendered file after batch 3 holds 7 entries (4 facts, 1 concept, 2 procedures) and withholds 2 claims the extractor had proposed. Reassessment also works in reverse: a deferred claim at e = 3 that meets one opposing trajectory falls to 2 and is left out of the next file, and one later supporting trajectory restores it to 3 (appendix C.2).
| What the paper measures | Reported | Source |
|---|---|---|
| Harmful entries, leave-one-entry-out | Mem0 11.7–13.4%, ACE 18.9–23.1%, SkillOpt 25.9–28.0%, Trace2Skill 26.0–32.7% across the DB, Cohort and Codebase streams | Table 1 |
| Aggregate score, GPT-5.4 | RIKA 0.232 vs ICL 0.183 (direct reuse of past trajectories) | Table 3 |
| Margin over the strongest baseline, per backbone | 0.046 to 0.081 across the four backbones | Section 5.2 |
| Best result per model and task stream | 14 of 24 combinations, not all | Section 5.2 |
| EKA added to other extractors, DB stream | SkillOpt 0.147 to 0.192; Trace2Skill 0.158 to 0.224 | Table 2 |
| Cost on GPT-5.4, main results | RIKA $25.89 vs ICL $16.32, Mem0 $16.43, ACE $62.35 | Table 20 |
Read the results as evidence for a design rule, not as a guarantee. The routing agent and the relation judge are model calls, so an error there can admit or block the wrong claim; the paper checks routing reliability in its appendix, and this article does not restate those numbers. The gain is also uneven: RIKA won 14 of 24 model-and-stream combinations, and several streams were won by other baselines. The reusable idea is narrower and sturdier. Treat the notes an agent writes about itself as unverified claims, and let each claim earn its place in context through evidence it can point to. That is a gate you can place in front of the structured-notes fix in The 4 Fixes, and it is a direct defense against the poisoning case in The 4 Failure Modes.
Goes deeper in: AI Agents → Context Engineering → The 4 Failure Modes and AI Agents → Context Engineering → The 4 Fixes
Related explainers
- CAPTURE — counterfactual memory auditing — audits a memory write by asking what else supports it at write time; RIKA instead keeps a claim pending and lets later tasks vote on it
- Matched replay gate for self-evolving skill memory — refuses a skill edit unless replayed cases still pass, which needs the extra task runs that RIKA avoids
- Randomized memory-slot exposure with inverse-propensity weighting — measures which memories help by deliberately randomizing what the agent sees