Latent Space

Dan Biderman on why AI needs a nervous system, not just notes

Dan Biderman· Co-founder and CEO of Engram at Engram
·~50 min·English·Latent Space
LLMAgentsInferenceAI InfrastructureTraining
TL;DR

Dan Biderman argues that long context and RAG alone can't give enterprise AI durable memory — context rot and the KV-cache make memory a systems problem — so models need compact knowledge trained into weights alongside text, not just longer context.

01Core Mental Model

Notes Aren't a Nervous System

Biderman's core claim is that today's LLMs behave like a cook who re-reads the recipe from scratch every time, while a real expert pairs notes with intuition stored in the weights.

if they can't write notes and they can not document the events of the day they would be uh in a disadvantage. But if you wipe their brain every evening, they would also be at a severe disadvantage.

— Dan Biderman, Latent Space
Key Insight
Biderman is careful not to dismiss text — he explicitly builds wikis and knowledge bases. The argument is additive: memory is the layer above notes, and it's the layer today's models are missing, so the fix is 'best of both worlds,' not replacing retrieval.
02The Limit

Context Rot

Long context windows don't solve memory, because the more you feed a model the more confused it gets — a failure that persists even at a 10-million-token window.

the more context you feed to the model, the more confused it gets. It's called the context rot.

— Dan Biderman, Latent Space
Key Insight
The subtle point is the gap between 'accepts the tokens' and 'reasons over them.' A model can ingest a huge context without erroring, yet still fail to hold it together — so raising the context ceiling treats the symptom, not the disease.
03The Systems Cost

The KV-Cache Monstrosity

Memory is a systems problem, not just a research one: on a Llama-70B, one Wikipedia article inflates into an 80 GB brain state, nearly as much GPU memory as the model's entire ~140 GB of parameters.

That's a systems problem. That's the the KV cache monstrosity that the smartest people in the world are trying to solve from the chip side of things

— Dan Biderman, Latent Space
Key Insight
Framing memory as a KV-cache cost reframes the whole product: if reading a corpus means re-paying that memory tax on every prefill, then training knowledge into compact weights becomes a powerful complement to context management as the economics get tighter at enterprise scale.
04The Approach

Cartridges: Knowledge You Load In

Engram's bet is to let a model study a corpus offline — quizzing itself and updating via gradient descent — into a compact brain-state 'cartridge' that is roughly a thousand times smaller and can be swapped in and out.

These are like capsules of knowledge you can load in and out of the model that are like a brain state that describes the model's world and the corpus in a way that's maybe a thousandx more compressed.

— Dan Biderman, Latent Space
Key Insight
This is prefill/decode economics inverted: instead of re-reading the corpus at inference, the expensive study happens once, ahead of time, so the model can start decoding almost immediately — aligning neatly with data centers that already split prefill and decode onto specialized hardware.
05The Scale

When Company Data Reaches Internet Scale

Biderman expects that within about 18 months many AI-native companies will hold trillions of tokens of internal data — internet-scale corpora that break text-only approaches on holistic questions.

I think that people don't fully comprehend the size of the of the knowledge workspaces they will deal with in 18 months um in 18 months many companies would have maybe trillions of tokens

— Dan Biderman, Latent Space
Key Insight
The tell is the holistic query RAG can't serve: 'which deals haven't we closed?' has no chunk that says 'not done' — you must absorb the whole corpus. Answered with frontier models plus compaction, that harmless-looking question can run into thousands of dollars, which is where trained-in memory earns its keep.
06The Open Problem

What Goes in the Weights, What Stays in Notes

Deciding which knowledge to internalize into weights versus externalize into text is an unsolved problem, and hand-picking the split turns into whack-a-mole across every user and company.

So the holy grail is have the model learn for itself. Have it operate with a notebook where it can take notes, have it operate with a with a brain associative parameter efficient thing that it can read from and have it decide when to go to each

— Dan Biderman, Latent Space
Key Insight
Biderman ties this straight back to human memory research — remembering everything is a known burden, so healthy forgetting is a feature. The ambition is to make the internalize-vs-externalize decision a learned behavior rather than a hand-tuned rule set.
07The End State

A Model You Own

The long-term vision is a personal set of weights that represents your knowledge, improves the more you use it, and belongs to you rather than to a shared provider model.

the more time they spend with the model, the better it gets for them. The more data they give the model, the better the model is for them. They control those sets of weights. It's theirs.

— Dan Biderman, Latent Space
Key Insight
Ownership is the quiet business thesis here. A thumbs-up to a shared provider maybe helps the next version for everyone; feedback to weights you control compounds only for you — which is exactly the lock-in a memory company would want to build.
08The Paradigm

Efficiency Is Intelligence

Biderman argues efficiency and intelligence can't be separated: the last era scaled by doing more with more, and the next era will do more with less to reach longer-horizon, harder tasks.

the principle is any kind of like efficiency and intelligence they cannot really be decoupled.

— Dan Biderman, Latent Space
Key Insight
This reframes 'efficient' away from 'cheap and second-rate.' If doing more with less is what lets you attempt tasks that were previously out of reach, then efficiency isn't a discount on intelligence — it's a path to more of it.