Chapter 91 · Memory and state

What an agent should remember, and for how long

Four kinds of memory with four different lifetimes, and the case for forgetting by default. Most sessions need none of it, and the ones that do need three specific things.

9 min read

Everything in section 7 assumed a run that starts empty and ends. That is the honest default, and this section is about the cases where it is the wrong one.

The question people ask first is "how do we give it memory?" The question worth asking first is what, specifically, is being lost by forgetting — because memory is storage you have to secure, invalidate, scope to a user, and explain. Added without a named thing it fixes, it is a liability with a plausible-sounding name.

Measure what forgetting actually costs

Before building anything, count.

-- How many sessions even have a second turn? Memory of any kind is
-- irrelevant to a single-turn session, and single-turn is usually the
-- overwhelming majority.
select turns,
       count(*) as sessions,
       round(100.0 * count(*) / sum(count(*)) over ()) as pct
  from (select session_id, count(*) as turns
          from question_log
         where asked_at > now() - interval '90 days'
         group by 1) t
 group by 1 order by 1;

Atlas, over 4,200 sessions in a quarter:

Turns	Sessions	Share
1	2,982	71%
2	714	17%
3–5	420	10%
6+	84	2%

Seventy-one percent of sessions end after one question. For those, every memory feature is dead weight — cost, risk and latency with no reader. That is the first thing a measurement buys you: the knowledge that this section applies to 29% of traffic, not to all of it.

The second thing is what the other 29% are actually doing. Reading 200 multi-turn sessions by hand, the follow-ups carried exactly three things forward:

Carried	Share of follow-ups	Example
The entity in focus	68%	"what about their credit terms?"
A filter already applied	24%	"same thing for last quarter"
An output preference	8%	"shorter this time"

Nothing in that list is a biographical fact about the user. It is the state of one conversation, and it expires when the conversation does.

Four kinds, four lifetimes

The word "memory" covers four mechanisms that share nothing but a metaphor.

Kind	Lifetime	Scope	What goes wrong
Working state	One step of the loop	One run	Nothing. It dies with the step
Conversation state	One session	One session	Stale pronouns, a filter that outlives its question
Durable memory	Weeks to years	One user	A fact that changed, remembered confidently
Organisational knowledge	Permanent	Everyone, by permission	This is retrieval, and calling it memory causes real confusion

The fourth row is the one that derails design meetings. "The assistant should remember our returns policy" describes a document in the corpus, reachable by anyone with permission, updated by editing the document. Storing it as memory creates a second copy with no owner and no update path — the exact failure that freshness exists to prevent.

If a fact belongs to the company, it is a document. If it belongs to the conversation, it is session state. Durable memory is the narrow remainder: things true about one person that no system of record holds.

The narrow remainder

That remainder is smaller than it sounds. For Atlas, after routing every candidate to the right mechanism:

// Candidates proposed in design, and where each actually belongs.
const ROUTING = {
  'the reps this user covers':      'system of record — the CRM knows',
  'the regions this user can see':  'permission model, chapter 13',
  'current returns policy':         'corpus — a document with an owner',
  'this user prefers short answers': 'durable memory',
  'this user works in US dollars':   'system of record — their profile',
  'last week this user asked about Northline': 'durable memory, maybe',
} as const

One clear survivor and one maybe. That is a normal result, and it is why this section spends as much time on what to refuse as on what to store.

The survivors share a property: nobody else owns them, and they are cheap to be wrong about. An output preference that goes stale produces a slightly-too-long answer. A credit limit that goes stale produces a wrong commitment to a customer. The first is a candidate for memory; the second belongs to the system that owns it, fetched fresh every time, as chapter 74 does it.

Giving every memory an expiry at the moment it is written

A memory with no expiry is a claim that something is permanently true, which is almost never what anyone meant.

// Memory is typed by what kind of claim it is, and each kind carries its own
// decay. The expiry is set at write time, by the writer, not inferred later
// by a cleanup job that has lost the context.
type MemoryKind =
  | 'preference'   // how they want answers. Stable, cheap to be wrong
  | 'context'      // what they were working on. Decays in days
  | 'correction'   // something they told us we got wrong. Valuable, fragile

interface Memory {
  id: string
  userId: string
  kind: MemoryKind
  statement: string          // one fact, in plain language
  sourceTurnId: string       // the turn that produced it, for audit
  writtenAt: Date
  expiresAt: Date
  supersededBy?: string      // set when a later memory contradicts it
}

// Defaults that came out of measuring how long each kind stayed true.
const TTL: Record<MemoryKind, number> = {
  preference: 180 * DAY,   // held 94% of the time at 180 days
  context: 7 * DAY,        // held 31% at 30 days, 88% at 7
  correction: 90 * DAY,    // the underlying document usually changes first
}

Those numbers came from a simple exercise: sample 100 memories of each kind, check whether each was still true at 7, 30, 90 and 180 days. context falling to 31% at thirty days is the finding that mattered — a month-old note about what someone was working on is wrong two times in three, and a wrong one is worse than an absent one because the model will use it.

The failure mode: the confidently remembered stale fact

A rep tells Atlas in March: "Northline is on 30-day terms now." Atlas stores it as a correction, because it contradicts a document. In August the rep asks about Northline's terms. Atlas answers 30 days, from memory, with no citation, in a confident tone.

Northline moved to 45-day terms in June. The contract document says so. Retrieval would have found it. Memory shadowed it.

Three things went wrong, and only the third is really about memory:

// 1. A correction was stored instead of being routed to the document owner.
//    The right handling of "the document is wrong" is a ticket, not a note
//    to self. Section 12 covers the feedback path that makes this possible.

// 2. Memory was allowed to answer without retrieval running. Memory should
//    bias retrieval, never replace it.

// 3. The memory outlived the fact and nothing noticed, because no check ever
//    compared a stored statement against the current corpus.

The third is chapter 96's whole subject, and it is the part most memory implementations skip.

The default this section argues for

Start with no durable memory at all. Ship conversation state, which is cheap and bounded and solves 92% of the measured follow-ups. Then add durable memory only for a named, measured gap, with a TTL set at write time and a check that proves it helps.

That ordering is not caution for its own sake. Durable memory is the feature that turns a stateless system you can reason about into a stateful one where a user's complaint depends on what the system wrote down three months ago, and chapter 99 covers what else it turns on.

Key points

Count single-turn sessions first. Atlas's were 71%, which bounds how much any memory feature can be worth.
Multi-turn follow-ups carried three things: the entity in focus, an applied filter, an output preference. None is biographical.
Four mechanisms wear the name memory: working state, conversation state, durable memory, and organisational knowledge. Only the third is new work.
A company fact is a document with an owner. Storing it as memory creates an uncontrolled copy.
Durable memory is the narrow remainder: true of one person, owned by no system, cheap to be wrong about.
Set the expiry at write time, by kind. Atlas's context memories were still true 31% of the time at thirty days.
Memory biases retrieval. It never replaces it, or it shadows the corrected document.
Ship conversation state first. It solved 92% of measured follow-ups at a fraction of the risk.

Next: conversation state inside one session — the cheap, bounded 92%.