Chapter 94 · Memory and state

Per-user memory, and keeping it separate

A memory store is a per-user datastore with no classification, filled by a model. The scoping key, the leak paths that are not the obvious one, and the test that runs on every build.

8 min read

Chapter 45 made the case that a retrieval filter is an access control, and that it has to be applied in the query rather than after it. Memory inherits all of that and adds a problem the corpus does not have: the corpus has a permission model, and memory has none.

A document arrived with an owner, a classification and an access list. A memory arrived as a sentence a model wrote down. Nobody labelled it, nobody reviewed it, and it may contain anything the user said.

The scoping key is a design decision

"Per user" is three different systems.

Key	What it means	Right when
`userId`	Memory belongs to one person	Preferences, corrections, working context
`teamId`	Memory is shared by a team	Shared workflows, team conventions
`userId + agentId`	Separate per assistant	Several assistants, different purposes

Atlas uses userId, and the decision deserves a sentence of justification rather than a default. A shared teamId memory means one rep's correction becomes another rep's silent context — with no notification, no attribution and no way for the second rep to know why the answer changed. Teams asked for it. The answer was that a team-level fact is a document, which is chapter 91's routing rule holding under pressure.

-- The key is on the table and on every index, so a query without it is slow
-- as well as wrong. A composite index whose first column is the tenant key
-- makes an unscoped query visibly expensive in development.
create table agent_memory (
  id            uuid primary key default gen_random_uuid(),
  user_id       text not null,
  kind          text not null check (kind in ('preference','context','correction')),
  statement     text not null,
  embedding     vector(1536),
  source_turn   uuid not null references question_log(id),
  written_at    timestamptz not null default now(),
  expires_at    timestamptz not null,
  superseded_by uuid references agent_memory(id)
);

create index on agent_memory (user_id, kind, expires_at);
create index on agent_memory using hnsw (embedding vector_cosine_ops);

Making the unscoped query impossible rather than discouraged

A filter that every caller must remember is a filter that one caller will forget. Postgres can enforce it below the application.

-- Row level security. The application connects as a role that cannot read
-- another user's rows even if the WHERE clause is missing entirely.
alter table agent_memory enable row level security;

create policy memory_owner on agent_memory
  using (user_id = current_setting('app.user_id', true));

// The setting is established per request, inside the same transaction as the
// query. Setting it on a pooled connection outside a transaction is the bug
// this pattern is famous for: the value leaks to whichever request gets that
// connection next.
export async function withUser<T>(userId: string, fn: (tx: Tx) => Promise<T>): Promise<T> {
  return db.transaction(async (tx) => {
    await tx.execute(sql`select set_config('app.user_id', ${userId}, true)`)
    return fn(tx)
  })
}

The true in set_config is the whole safety property: it scopes the setting to the transaction, so it is discarded when the transaction ends. With false it persists on the connection and the next request inherits it. That is a one-character difference between an access control and a cross-user leak, which is why it gets a comment in the code and a test below.

The leak paths that are not the query

Scoping the store is the easy part. Four paths carry memory across users without any query being wrong.

// 1. Shared caches keyed on the question alone. Two users, same question,
//    different memories in context — and the second gets the first's answer.
const key = `answer:${hash(question)}`               // leaks
const safe = `answer:${userId}:${hash(question)}`    // scoped

// 2. Embedding caches. The embedding of a memory statement is derived from
//    its text, and a shared embedding cache keyed on text is fine. A cache
//    keyed on memory *id* across a shared namespace is not, because the id
//    resolves to content.

// 3. Evaluation and debugging fixtures. A trace captured from one user's
//    session, saved as a test case, now holds their memories in a repo that
//    a wider group can read. Chapter 89's trace redaction applies to the
//    recalled-memory block exactly as it applies to retrieved passages.

// 4. Subagents. Chapter 84's subagent gets a task description assembled by
//    the parent. If the parent pastes recalled memories into it and the
//    subagent runs under a service identity, the memory has left the user's
//    scope and the subagent's own trace now carries it.

The fourth is the one that surprised people on review, because the subagent design looked like a pure refactor of the loop. Chapter 84 makes the general version of the point: identity has to travel with the task, and the task description is content, not metadata.

A test that runs on every build

Separation is a property that is cheap to assert and catastrophic to lose, which is the definition of something that belongs in the suite.

// Two users, two memories, four assertions. It has caught two regressions:
// a cache key that lost its prefix in a refactor, and a recall helper that
// took userId as an optional parameter.
test('memory never crosses users', async () => {
  await writeMemory(alice, { kind: 'preference', statement: 'prefers short answers' })
  await writeMemory(bob, { kind: 'context', statement: 'works with the Northline account' })

  const forAlice = await recallMemories(alice, 'tell me about Northline')
  expect(forAlice.map((m) => m.statement)).not.toContain('works with the Northline account')

  // The same question, answered twice, must not be served from one cache entry.
  const a = await answer(alice, 'what do you know about me?')
  const b = await answer(bob, 'what do you know about me?')
  expect(a.text).not.toContain('Northline')
  expect(b.text).not.toContain('short answers')

  // And the store itself refuses, with no WHERE clause at all.
  await expect(withUser(alice, (tx) => tx.execute(sql`select * from agent_memory`)))
    .resolves.toHaveLength(1)
})

The last assertion is the one that proves the policy rather than the application. A test that only exercises recallMemories passes just as happily when the row-level policy has been dropped.

Deletion has to be real

A user who asks to be forgotten, or who leaves the company, creates an obligation that chapter 17 already described in the abstract. Memory is where it gets concrete, because memory is the one store holding content the user authored about themselves.

// Hard delete, and everywhere it propagated. A soft delete here is the wrong
// default: the row's whole content is personal, so tombstoning it keeps
// exactly the thing the request was about.
export async function forgetUser(userId: string) {
  await withUser(userId, (tx) => tx.execute(sql`delete from agent_memory where user_id = ${userId}`))
  await redis.del(`session:*:${userId}`)
  await redis.del(`answer:${userId}:*`)
  await scrubTraces(userId)            // chapter 89's traces carry the recalled block
  await auditLog.record('memory.forget', { userId, at: new Date() })
}

The audit entry records that the deletion happened and holds no memory content. That is the shape every one of these operations wants: provable, and empty.

Key points

The corpus has a permission model; memory has none. A memory is an unlabelled sentence a model wrote about whatever the user said.
Choose the scoping key deliberately. Team-shared memory makes one person's correction another's silent context, with no attribution.
Enforce the scope in the database with row level security, so a missing WHERE clause fails rather than over-returning.
Scope the setting to the transaction. On a pooled connection, a non-local setting leaks to the next request.
Four leak paths bypass the query entirely: shared answer caches, id-keyed caches, saved traces and fixtures, and subagent task descriptions.
Test separation on every build, including one assertion against the store with no filter, so the policy is proved rather than the helper.
Deletion is a hard delete across the store, the caches, the traces and the sessions. The audit entry proves it happened and carries nothing.

Next: summarising a conversation without losing the thread — compression, and what it quietly throws away.