Chapter 93 · Memory and state

Long-term memory across sessions

Writing something down that outlives the conversation. What qualifies, who decides, how it gets retrieved, and the measurement that tells you whether it helped at all.

8 min read

Chapter 92 handled the 92%. This chapter is the remainder: something learned in one conversation that should still be true in the next one.

The design question is not storage. It is who decides what gets written, and there are only three answers.

Three ways a memory gets written

Writer	How it works	Precision	Recall
The user, explicitly	"Remember that I cover the western region"	~1.00	Very low
The model, per turn	A model call decides what was memorable	0.58	High
A rule, on specific events	A correction, a stated preference, a repeated filter	0.91	Moderate

Atlas uses the third, with the first available as an escape hatch. The second was built, measured and removed, and the measurement is worth showing because model-written memory is the default design in most tutorials.

// The version that was removed. After each turn, ask the model what is worth
// remembering. It is one extra cheap call and it sounds obviously right.
const EXTRACT_PROMPT = `
From the exchange below, list any durable facts about this USER that would
help answer their future questions. Output JSON: {"memories": ["..."]}.
Output an empty list if there is nothing durable. Most exchanges have nothing.
`

Over 900 turns it wrote 412 memories. Hand-reviewed:

Verdict	Count	Share
Genuinely durable and useful	94	23%
True, useless	186	45%
About the company, not the user	98	24%
Wrong	34	8%

"True, useless" is the bulk of it: "The user asks about orders", "The user works with customers". Harmless alone, and 186 of them crowd out the 94 that matter. The 24% about the company are chapter 91's routing failure, written by a model that has no way to know what the corpus already holds. The 8% wrong are the expensive ones.

Telling the model to be stricter moved the volume and not the ratio. The model has no access to the thing that decides whether a fact is worth storing — whether any other system already owns it.

Rules that fire on events

// Three events, each of which is already detected for another reason.
export function memoryCandidates(turn: Turn, state: SessionState): Draft[] {
  const out: Draft[] = []

  // 1. A correction. The user told you an answer was wrong and said why.
  //    Chapter 63's abstention path and the feedback control in section 12
  //    both surface this already.
  if (turn.feedback?.kind === 'incorrect' && turn.feedback.comment) {
    out.push({ kind: 'correction', statement: turn.feedback.comment, confidence: 0.9 })
  }

  // 2. An explicit preference, stated imperatively about future answers.
  const pref = matchPreference(turn.question)   // "always", "from now on", "I prefer"
  if (pref) out.push({ kind: 'preference', statement: pref, confidence: 0.95 })

  // 3. A filter applied in three separate sessions. Repetition is evidence
  //    that no single session can give you.
  const repeated = repeatedAcrossSessions(turn.userId, state.filters, { minSessions: 3 })
  for (const r of repeated) {
    out.push({ kind: 'context', statement: `Usually works with ${r.key}=${r.value}`, confidence: 0.7 })
  }

  return out
}

Over the same 900 turns these rules wrote 68 memories, of which 62 were durable and useful. Precision 0.91, against 0.23. The recall is lower in the sense that fewer memories exist, and roughly a third of the model's 94 good ones were missed.

That trade is the right way round here. A missing memory costs one slightly worse answer. A wrong memory is asserted confidently, is invisible to the user, and persists.

Retrieving memory, which is retrieval

A stored memory that is always in the prompt is part of the system prompt with worse change control. Once there are more than a handful, memory has to be selected per question — which makes it the same problem section 4 already solved.

// Same hybrid machinery as the corpus, against a much smaller index. The
// user filter is a hard pre-filter, never a ranking signal — chapter 45's
// rule, and the consequences of getting it wrong are worse here.
export async function recallMemories(userId: string, question: string): Promise<Memory[]> {
  const hits = await hybridSearch({
    query: question,
    index: 'memories',
    filter: { userId, expiresAt: { gt: new Date() }, supersededBy: null },
    limit: 12,
  })
  // Preferences are small, few, and relevant to every answer's shape rather
  // than its content, so they bypass ranking and are always included.
  const prefs = await activePreferences(userId)
  return dedupe([...prefs, ...hits.filter((h) => h.score > 0.45)]).slice(0, 6)
}

The score > 0.45 threshold and the cap of six both exist for the same reason: an irrelevant memory in context behaves exactly like an irrelevant chunk, and chapter 61 measured what that costs. Uncapped recall put a median of 11 memories into the prompt and cut groundedness by four points.

Rendering it so the model treats it correctly

// Memories are marked as recalled and dated. Without the marking, the model
// blends them with retrieved passages and cites them like documents.
const block = memories.length
  ? `## Recalled about this user
${memories.map((m) => `- (${m.kind}, noted ${fmtDate(m.writtenAt)}) ${m.statement}`).join('\n')}

These are notes from earlier conversations with this user. They are not
sources. Do not cite them. If one conflicts with a retrieved document, the
document wins and you should say that the note appears to be out of date.`
  : ''

That last sentence is the fix for chapter 91's stale-Northline failure. Instructed this way, Atlas answered from the document and added "a note from March said 30 days, which no longer matches the contract" — which is better than both the silent wrong answer and the silent correct one, because it tells the user their own earlier information was wrong.

Does it actually help?

The whole feature needs a number, and the honest one is hard to get: you cannot A/B a user against themselves on the same question.

// What worked: hold out memory on a random 10% of eligible turns, where an
// eligible turn is one where recall returned at least one memory. Compare
// answer quality on those turns only. Comparing all turns dilutes the effect
// into nothing, which is how this feature gets declared a success.
const useMemory = !(hashTurn(turnId) % 10 === 0)

Atlas, over eight weeks, 1,340 eligible turns:

Metric	With memory	Held out
Answer rated helpful	78%	71%
Follow-up needed	19%	27%
Groundedness	0.96	0.96
Wrong answers traced to a memory	1.1%	0%

Seven points of helpfulness and eight fewer points of follow-up, against 1.1% of answers made wrong by a stored note. On this corpus that was worth keeping. It is a judgement with a number attached rather than an assumption, and a team with higher stakes per answer could read the same table and decide the other way.

Note the eligibility filter doing its work: across all turns, the helpfulness difference was 1.2 points and would have looked like noise.

Key points

The design question is who writes a memory, and there are three answers: the user, the model, or a rule on specific events.
Model-written memory scored 0.23 precision over 900 turns: 45% true-but-useless, 24% company facts, 8% wrong. Stricter prompts moved volume and not the ratio.
Rules on corrections, stated preferences and filters repeated across three sessions scored 0.91 at a third of the volume.
A missing memory costs one worse answer. A wrong memory is confident, invisible and persistent. Trade recall for precision.
Once there are more than a handful, memory must be retrieved per question with the same hybrid machinery, and the user filter is a hard pre-filter.
Cap recall. Uncapped put a median of 11 memories in context and cost four points of groundedness.
Mark memories as notes, date them, forbid citing them, and state that a retrieved document wins a conflict.
Measure by holding memory out on eligible turns only. Across all turns the effect disappears into noise.

Next: per-user memory, and keeping it separate — the pre-filter above, treated as the security control it is.