Chapter 95 · Memory and state

Summarising a conversation without losing the thread

Compression is lossy by definition, so the question is what you choose to lose. Rolling summaries, the drift they accumulate, and keeping the parts that cannot be paraphrased.

8 min read

Chapter 92 kept sessions bounded with three slots and avoided summarisation entirely, which works for the 98% of Atlas sessions under six turns. This chapter is about the other case: a long investigation, a resumed run, or an agent loop whose step history has outgrown the window.

When compression becomes unavoidable, one principle decides every design detail: you choose what to lose, or the summariser chooses for you.

What a naive rolling summary loses

// The standard approach. Every N turns, replace the oldest turns with a
// summary of them, and carry the summary forward.
async function roll(history: Turn[], summary: string): Promise<string> {
  return generate(`Summarise this conversation so far, including the previous
summary. Be concise.

PREVIOUS SUMMARY: ${summary}
NEW TURNS: ${render(history)}`)
}

Run on 40 twelve-turn Atlas investigations, then checked for which facts survived:

Fact type	Present at turn 12
Topic of the conversation	100%
Identifiers (order numbers, account codes)	34%
Exact figures	51%
Constraints stated by the user	62%
Actions already taken	41%
Things explicitly ruled out	22%

The top row is what makes this failure hard to notice: the summary always reads correctly. It is fluent, on-topic and plausible. What it drops is identifiers, numbers and negatives — precisely the things that cannot be recovered by inference, because an identifier has no semantic neighbours and a ruled-out option looks like an unexplored one.

The last row is the expensive one. A twelve-turn investigation that established "it is not a credit hold" on turn three has, by turn twelve, a summary that does not say so, and the agent proposes checking credit holds again. Chapter 90's loops that revisit their own ground are often this, rather than a termination bug.

Summarisation preserves what can be paraphrased and discards what cannot. Identifiers, figures and negatives are exactly the content with no paraphrase.

Structured compression

The fix is to stop asking for a summary and start asking for a record with fields, keeping the unparaphrasable parts verbatim outside the model's reach.

// The summariser fills prose fields. Everything that must survive exactly is
// collected mechanically and never passes through generation.
interface ConversationRecord {
  narrative: string              // model-written, 3-4 sentences
  entities: Entity[]             // extracted by tool results, verbatim
  figures: { label: string; value: string; turnId: string }[]
  constraints: string[]          // user-stated, quoted verbatim
  actionsTaken: ActionRecord[]   // from the action log, not from text
  ruledOut: { claim: string; evidence: string; turnId: string }[]
  openQuestions: string[]
}

// Entities, figures and actions come from structured sources that already
// exist. Chapter 92 captures entities from tool results; chapter 73's
// idempotency records carry every action; chapter 57's schemas hold figures.
function mechanicalFields(turns: Turn[]): Pick<ConversationRecord,
  'entities' | 'figures' | 'actionsTaken'> {
  return {
    entities: dedupeById(turns.flatMap((t) => t.toolResults.flatMap(entitiesIn))),
    figures: turns.flatMap((t) => t.structured?.figures ?? []),
    actionsTaken: actionLog.forSession(turns[0].sessionId),
  }
}

Same 40 investigations, recompiled this way:

Fact type	Naive	Structured
Identifiers	34%	100%
Exact figures	51%	100%
Constraints	62%	94%
Actions taken	41%	100%
Ruled out	22%	89%

The three rows at 100% are the ones that never touch the model. That is the whole mechanism, and it generalises past summarisation: anything that must survive exactly should be carried by a data structure rather than regenerated from text.

ruledOut at 89% still depends on extraction, because "it is not a credit hold" is a conclusion in prose. It is prompted for explicitly and verified against the turn it cites, which is where the remaining 11% is lost.

Compounding drift

A rolling summary summarises a summary, and errors compound in one direction.

// Measured by seeding a conversation with 20 known facts and compressing 10
// times with no new content. Pure loss, with nothing added.

Compression	Facts retained	New claims not in the original
1	19/20	0
3	16/20	1
5	13/20	3
10	9/20	7

Fact loss is expected. The right-hand column is the one to look at: by the tenth pass the record asserts seven things that were never said. Each pass smooths an uncertainty into a statement, and the next pass treats that statement as given. Chapter 83 found the same shape in self-critique, for the same reason — a model asked to rewrite its own output finds something to change.

// Two rules that bound it.

// 1. Never summarise a summary. Recompile from the original turns each time,
//    which is affordable because turns are stored anyway and the compile is
//    one cheap call.
const record = await compile(allTurns)        // not: compile(record, newTurns)

// 2. Cap total compressions per session. Past the cap, the session is too
//    long to carry honestly — start a new one and hand over the record
//    explicitly, where the user can see what was kept.
if (session.compressions > 6) return handOver(record)

Recompiling from source is what makes the first rule affordable, and it is the same argument chapter 34 made about rebuilding from the source of truth rather than patching derived state.

When to compress

// Token pressure is the obvious trigger and the wrong one on its own: it
// fires mid-investigation, exactly when the detail matters most.
const shouldCompress =
  tokens > 0.6 * WINDOW ||          // pressure
  turnsSinceCompression > 15 ||     // length
  topicChanged(state, question)     // a natural seam

The third trigger is the valuable one. Compressing at a topic change loses almost nothing, because the closed topic is genuinely finished; compressing mid-thread at 60% window pressure loses the thread being pulled. Atlas compresses on a seam 71% of the time, and those compressions retain nine points more of the ruledOut set than pressure-triggered ones.

Showing the user what was kept

// A long session displays a one-line marker at the compression point, which
// expands into the record. Two benefits, and the second was unexpected.
<CompressionMarker turn={n} record={record} />

The first is correction: a user who sees "ruled out: credit hold" when it was never ruled out can fix it in one click.

The second is that it made long sessions legible. Users started opening the record to see where an investigation had got to, treating it as a working summary of their own. Atlas's markers are expanded in 23% of long sessions, against the 4% click-through that was expected from chapter 62's baseline. The feature built for correction mostly gets used for orientation.

Key points

A naive rolling summary keeps the topic and drops identifiers (34%), actions taken (41%) and things ruled out (22%). It always reads correctly, which is why the loss is invisible.
Summarisation keeps what can be paraphrased. Identifiers, figures and negatives have no paraphrase.
Compile a structured record instead. Carry entities, figures and actions from existing structured sources so they never pass through generation, and they survive at 100%.
Compressing a compression compounds. Ten passes lost 11 of 20 facts and invented seven that were never said.
Recompile from the original turns every time, and cap compressions per session. Past the cap, hand over visibly.
Compress on a topic seam rather than on token pressure. Seam compressions retained nine more points of the ruled-out set.
Show the record at the compression point. Atlas's markers are opened in 23% of long sessions, mostly for orientation rather than correction.

Next: memory invalidation, and the fact that changed — the stale note from chapter 91, dealt with properly.