Chapter 100 · Memory and state

Testing that memory helps rather than drifts

Memory is the one feature that can degrade a system slowly while every test passes. Multi-session fixtures, the drift metrics worth watching, and the kill criteria agreed before launch.

8 min read

Every other feature in this series fails loudly. A broken retriever returns nothing; a broken tool throws; a broken loop runs forever and hits its step limit.

Memory fails quietly and gradually. It accumulates wrong notes one at a time, each of which degrades a few answers, and no individual answer is bad enough to report. Chapter 90 listed seven bugs that shipped with a green suite because testing could not supply concurrency or elapsed time. Memory's version is worse: the thing testing cannot supply is accumulated history, and the system is designed to accumulate it.

Fixtures that span sessions

A single-turn fixture cannot exercise memory at all. The unit of test is a sequence of sessions with time between them.

// A scenario is a scripted history plus the question that should benefit,
// with a clock the test controls. Chapter 90's lesson about elapsed time
// applies: you cannot wait three weeks, so the clock is an input.
interface MemoryScenario {
  id: string
  sessions: { atDays: number; turns: { question: string; expectMemoryWrite?: Partial<Draft> }[] }[]
  probe: { atDays: number; question: string }
  expect: {
    recalls?: string[]        // statements that must be in context
    excludes?: string[]       // statements that must not be
    answerContains?: string[]
  }
}

// The scenario that encodes chapter 91's stale-Northline failure, as a test.
const STALE_FACT: MemoryScenario = {
  id: 'corpus-supersedes-stale-memory',
  sessions: [
    { atDays: 0, turns: [{
        question: 'Northline is on 30-day terms now, your answer was wrong',
        expectMemoryWrite: { kind: 'correction' },
    }]},
  ],
  probe: { atDays: 150, question: 'what are Northline’s payment terms?' },
  expect: {
    answerContains: ['45', 'contract'],
    excludes: ['30-day terms'],          // the note must not win
  },
}

The fixture needs a corpus that changed between day 0 and day 150, which means the test corpus has to be versioned by date. That is the part teams skip, and it is the part that makes the test real: without it, the memory and the document never disagree and the scenario proves nothing.

Atlas runs 34 scenarios. Eleven are adversarial, drawn directly from chapter 99 — an instruction phrased as a preference, a memory candidate sourced from a retrieved document, a recalled statement appearing in a tool argument.

The four drift metrics

Scenarios catch known failures. Drift is the unknown one, and it needs production measurement.

Metric	What it detects	Atlas's band
Memories per active user	Unbounded accumulation	4–14
Share of answers using recall	Over- or under-reach	18–31%
Deletion rate per 100 recalls	Precision falling	< 2.5
Contradiction rate in the nightly sweep	The corpus moving under the store	< 2%

-- All four from one query, weekly, so a trend is visible before a threshold
-- is crossed. A single week's value means very little on its own.
select date_trunc('week', d) as week,
       round(avg(memories_per_user), 1)                   as per_user,
       round(100.0 * avg(recall_used))                    as recall_pct,
       round(100.0 * sum(deletions) / nullif(sum(recalls), 0), 2) as del_per_100,
       round(100.0 * avg(contradicted_share), 2)          as contradicted_pct
  from memory_daily
 where d > now() - interval '16 weeks'
 group by 1 order by 1;

The third row is the most sensitive, and it is the user telling you the precision estimate from chapter 93 has moved. Atlas's deletion rate went from 1.4 to 3.9 per 100 recalls over three weeks after a change to the preference rule, which was the signal that caught the over-generalisation problem before the quarterly review did.

The first row catches the thing nobody watches for: a user who accumulates 60 memories reaches chapter 97's cliff, where preference-following falls to 54%. A per-user cap with oldest-first eviction within kind bounds it, and the metric proves the cap is working.

The held-out comparison, run continuously

Chapter 93 used a 10% hold-out to decide whether memory was worth shipping. The mistake is to run it once.

// Keep it on permanently. A feature whose value is measured by comparison
// needs the comparison to continue existing, and 10% of eligible turns is
// a cheap insurance premium against a silent regression.
const useMemory = !(hashTurn(turnId) % 10 === 0)

Atlas's eight-week margin was seven points of helpfulness. Watched over the following quarter:

Period	Margin	Note
Weeks 1–8	+7.0	Launch measurement
Weeks 9–12	+6.4	Stable
Weeks 13–16	+2.1	Preference-rule change
Weeks 17–20	+6.9	After the fix

The dip in weeks 13–16 is the over-generalisation period. The deletion-rate metric moved first, in week 13; the helpfulness margin confirmed it by week 15. Neither would have been visible without a hold-out that was still running four months after the decision it was built for.

Kill criteria, written before launch

The uncomfortable part, and the reason to write it down while the feature is still a proposal.

// Agreed before shipping. Each is a measurable condition with an action
// attached, so the decision is made once rather than argued under pressure.
const KILL_CRITERIA = [
  { metric: 'helpfulness margin', threshold: '< +2 points for 4 consecutive weeks',
    action: 'disable durable memory; keep session state' },
  { metric: 'wrong answers traced to a memory', threshold: '> 3% of recall-using answers',
    action: 'disable durable memory immediately' },
  { metric: 'deletion rate', threshold: '> 8 per 100 recalls for 2 weeks',
    action: 'freeze writes, review the rules' },
  { metric: 'any confirmed cross-user recall', threshold: 'one occurrence',
    action: 'disable and audit' },
] as const

The last one has no tolerance for a reason: it is the only entry whose failure harms someone other than the person asking the question.

Writing these down in advance is cheap and almost never done. The argument for it is that memory is the feature most likely to be defended on narrative rather than number — it is intuitive, it demos well, and nobody wants to remove something users were told the assistant could do. A threshold agreed before anyone was attached to the feature is the only version of that conversation worth having.

Memory is the one component that can make a system worse while every test is green and every user is mildly satisfied. The hold-out is not a launch artifact. It is the monitor.

Key points

Memory is the only feature that degrades gradually with a green suite, because what testing cannot supply is accumulated history.
The unit of test is a sequence of sessions with a controlled clock, against a corpus versioned by date, so the memory and the document can actually disagree.
Keep adversarial scenarios in the same suite. Atlas runs 11 of 34 straight from the attack chapter.
Watch four drift metrics weekly: memories per user, share of answers using recall, deletions per 100 recalls, and the sweep's contradiction rate.
The deletion rate moves first. It caught an over-generalisation regression two weeks before the helpfulness margin did.
Cap memories per user. Past roughly thirty, preference-following collapses and memory is present and ignored.
Run the held-out comparison permanently. Atlas's margin fell from 7.0 to 2.1 and recovered, and only a running hold-out showed it.
Write kill criteria before launch, including one with zero tolerance: any confirmed cross-user recall.

That closes section 8. Memory is now bounded, scoped, invalidated, visible and measured — and the measurement is the part the rest of the series leans on.

Next, and the start of section 9: evaluation. Everything so far has been measured with numbers produced ad hoc for the question at hand. Section 9 builds the harness properly — what to measure, how to build a test set that survives contact with production, the judge and its calibration, regression gates, and how to tell a real improvement from noise.