Chapter 96 · Memory and state

Memory invalidation, and the fact that changed

A stored note has no way to know the world moved. Expiry, supersession, contradiction detection against the corpus, and why the document always wins.

8 min read

Chapter 91 left a note stored in March answering a question in August, five months after the contract changed. This chapter is the machinery that prevents it, and it is four mechanisms rather than one.

The framing worth holding: a memory is a cached derivation of a world that kept moving. Everything cache invalidation is hard about applies, plus the memory is prose, has no key, and the underlying fact never notified anyone.

Four mechanisms, cheapest first

Mechanism	Catches	Cost	Latency to catch
Expiry	Everything, eventually	Free	The TTL
Supersession	A later statement contradicting an earlier one	Free	Next write
Corpus contradiction	A document that changed under the memory	One embedding search per memory	Daily sweep
Use-time check	The remainder	One model call, sampled	At answer time

They stack. Expiry alone is a blunt instrument that deletes good memories and keeps bad ones for up to their TTL; it is still the one to build first, because it bounds the damage of every bug in the other three.

Expiry that is actually enforced

// The filter lives in recall, not in a cleanup job. A cleanup job that fails
// silently turns the TTL off, and nothing in the answer path notices.
filter: { userId, expiresAt: { gt: new Date() }, supersededBy: null }

-- The job exists too, for storage rather than correctness. Expired rows are
-- deleted rather than archived: the content is personal, chapter 94's
-- deletion argument applies, and nothing downstream reads them.
delete from agent_memory where expires_at < now() - interval '7 days';

The seven-day grace is there so a wrongly-short TTL shows up as a recoverable support question rather than as data that no longer exists.

Supersession

// On every write, look for an existing memory that this one contradicts.
// Same user, same kind, high similarity. The old row is marked rather than
// deleted, because "what did it think last month" is a question the audit
// trail has to answer.
export async function writeMemory(userId: string, draft: Draft) {
  const near = await hybridSearch({
    query: draft.statement,
    index: 'memories',
    filter: { userId, kind: draft.kind, supersededBy: null },
    limit: 3,
  })

  const id = await insert({ ...draft, userId, expiresAt: expiryFor(draft.kind) })

  for (const m of near) {
    if (m.score > 0.82 && (await contradicts(m.statement, draft.statement))) {
      await update(m.id, { supersededBy: id })
    }
  }
  return id
}

The 0.82 threshold narrows the contradiction check to a handful of candidates, which keeps it affordable. contradicts is a cheap model call with a narrow task, and chapter 62's entailment finding applies: narrow tasks get high agreement with human raters. On 200 hand-labelled pairs it scored 0.93.

Supersession fires on one path only — a new memory arriving. It cannot catch the common case, where nothing new was said and the world changed anyway.

Contradiction against the corpus

This is the mechanism that would have caught Northline, and it is a sweep rather than an event.

// For each live memory, retrieve what the corpus now says about it and ask
// whether the document contradicts the note. Cheap enough to run nightly
// because the memory store is small — Atlas holds about 3,100 live rows.
export async function sweep() {
  for await (const m of liveMemories()) {
    const docs = await hybridSearch({ query: m.statement, index: 'corpus', limit: 3 })
    if (!docs.length || docs[0].score < 0.5) continue      // nothing relevant; leave it

    const verdict = await checkContradiction(m.statement, docs[0].content)
    if (verdict === 'contradicted') {
      await update(m.id, { expiresAt: new Date(), contradictedBy: docs[0].id })
      await notify(m.userId, { memory: m, document: docs[0] })
    }
  }
}

// The prompt. It has to be strict about the difference between "the document
// does not mention this" and "the document says otherwise" — the first is
// the overwhelmingly common case and treating it as contradiction empties
// the store.
const CONTRADICTION_PROMPT = `
Does the DOCUMENT contradict the NOTE? Answer exactly "contradicted",
"consistent", or "unrelated".

"contradicted" requires the document to state something that cannot be true
at the same time as the note. A document that is silent on the note's subject
is "unrelated", never "contradicted".

NOTE: {{note}}
DOCUMENT: {{document}}
`

Atlas's first full sweep over 3,100 memories:

Verdict	Count	Share
unrelated	2,588	83.5%
consistent	468	15.1%
contradicted	44	1.4%

Forty-four live memories were contradicted by the current corpus, of which 39 were genuine on review. The oldest had been wrong for four months and had been used in eleven answers.

The notification matters as much as the expiry. A user told "a note from March about Northline's terms no longer matches the contract" learns something about their own understanding, which is worth more than the correction to the assistant.

The use-time check

// Last line, at answer time, for the memories that actually made it into
// context. Sampled, because it costs a call per recalled memory.
if (shouldSample(0.05) && recalled.length) {
  void verifyRecalled(recalled, retrievedChunks, answerId)   // off the hot path
}

Running it on every answer was measured and rejected: it added 340 ms at p50 and caught 0.2% more than the nightly sweep, because the sweep had already removed almost everything the check would find. Sampling it keeps a drift signal without paying for it on every request.

The rule that makes all of this survivable

// Mechanisms fail. The instruction in chapter 93's memory block is the
// backstop that holds when all four have missed something.
//
//   "If a note conflicts with a retrieved document, the document wins
//    and you should say the note appears to be out of date."

This is worth stating as a principle rather than a prompt line. Memory and the corpus are two claims about the world with different provenance: one has an owner, a review process and a change history; the other is a sentence written by a model after a conversation. When they disagree, the ranking is not close, and encoding it once means every invalidation gap degrades into a mildly worse answer rather than a wrong one.

Every invalidation mechanism is a race against the world changing. The ordering rule is what makes losing that race survivable.

What this costs

// The whole invalidation stack, per month, at Atlas's scale:
//   nightly sweep: 3,100 memories x 1 search + 1 cheap call  = ~$9
//   supersession:  ~70 writes x up to 3 contradiction checks = ~$0.40
//   use-time:      5% of 1,340 eligible turns                = ~$1.20

About ten dollars a month to keep a memory store honest, against a feature worth seven points of helpfulness. That ratio is the argument for building all four rather than picking one, and it is the kind of number worth putting in front of whoever asks why memory needs a nightly job.

Key points

A memory is a cached derivation of a world that kept moving, with no key and no invalidation signal.
Four mechanisms stack: expiry, supersession on write, a nightly corpus contradiction sweep, and a sampled use-time check.
Enforce expiry in the recall filter, not in a cleanup job. A failed job must not silently disable the TTL.
Supersede rather than delete. "What did it believe last month" is an audit question.
The sweep is the one that catches a fact that changed with nothing new being said. Atlas's first sweep contradicted 1.4% of live memories; the oldest had been wrong four months and used eleven times.
Make the prompt distinguish silence from contradiction. Treating a silent document as contradiction empties the store.
Notify the user, not just the system. The correction is worth more to them than to the assistant.
The backstop is an ordering rule: the document wins, and the answer says the note looks out of date.

Next: storing memory: the options and their trade-offs — where these rows actually live.