Chapter 121 · Guardrails and security

Preventing data leakage between users

Every check in the last two chapters can be correct and data still crosses. Eight paths that bypass the query entirely, the key-prefix discipline, and a test that runs on every build.

8 min read

Chapter 119 put the filter in the query. Chapter 120 threaded identity to every tool. Both can be correct and one user can still see another's data, because the leak paths do not go through a query.

Chapter 94 covered four of these for memory. This is the complete list for the system, and the point of enumerating them is that each one is invisible from the code that does the right thing.

Eight paths

// 1. An answer cache keyed on the question.
const key = `answer:${hash(question)}`                 // ❌
const key = `answer:${userId}:${hash(question)}`       // ✅

Two users ask the same question, have different permissions, and the second gets the first's answer. It is the most common leak in this class because caching answers is such an obvious optimisation — Atlas's hit rate was 23%, which is exactly enough to make it tempting.

// 2. A retrieval cache keyed on the query embedding. Same shape, one
//    layer down, and harder to spot because the key looks like a vector
//    rather than like user-facing text.

// 3. Memory recall without a hard pre-filter. Chapter 93's rule: the
//    user filter is a WHERE clause, never a ranking signal.

// 4. Shared conversation state. A session store keyed on a conversation
//    id that is guessable, or reachable without an ownership check.
const state = await redis.get(`session:${sessionId}`)            // ❌ no owner check
const state = await loadSession(sessionId, ctx.userId)           // ✅

// 5. Traces, saved as test fixtures. Chapter 89's trace carries the
//    retrieved passages and the recalled memories. Committed to a repo
//    a wider group can read, it has left the permission boundary.

// 6. Error messages and debug output. A stack trace or a "no results
//    for document d-9912" tells the user a document exists.

// 7. Subagents under a service identity. Chapter 84's task description is
//    content, assembled by a parent that may have read restricted data.

// 8. Aggregates. "The average margin across all accounts is 14%" uses
//    data the asker cannot see, and no per-document check fires, because
//    no document is quoted.

The eighth is the one with no mechanical fix, and it is covered at the end.

Prefixing as a discipline rather than a convention

// Every cache key in the system is produced by one function. It is
// impossible to construct a key without an owner, because the type has
// no other constructor.
export type ScopedKey = string & { readonly __scoped: unique symbol }

export function scopedKey(ctx: CallerContext, namespace: string, discriminator: string): ScopedKey {
  return `${namespace}:u=${ctx.userId}:g=${hashGroups(ctx.groups)}:${discriminator}` as ScopedKey
}

// And the cache accepts nothing else.
export const cache = {
  get: <T>(k: ScopedKey) => store.get<T>(k),
  set: <T>(k: ScopedKey, v: T, ttl: number) => store.set(k, v, ttl),
}

The group hash in the key is the detail that is usually missed. A user whose access narrows keeps a cache entry computed when it was wider, so the key has to change when the permissions do. Hashing the group list achieves that with no invalidation logic: the old entry becomes unreachable and expires on its own.

Atlas's cache hit rate fell from 23% to 11% when keys became user-scoped. That is the honest cost, it was accepted without argument, and it is worth quoting to anyone proposing a shared answer cache.

Traces and fixtures

// A trace is a complete record of what the model saw. Saving one is
// copying restricted data into a new location with different access
// control, and that decision has to be explicit.
export function redactTrace(t: Trace, level: 'debug' | 'share' | 'fixture'): Trace {
  if (level === 'debug') return t                      // stays in the user's own view
  return {
    ...t,
    context: t.context.map((c) => ({ ...c, content: level === 'share' ? '[redacted]' : synthetic(c) })),
    recalledMemories: [],
    toolResults: t.toolResults.map(redactValues),
  }
}

// Fixtures get synthetic content rather than redaction, because an eval
// case needs text that behaves like the original. Chapter 102's growth
// path — complaints becoming cases — runs through this function, and it
// is the step that makes that practice safe.

Error messages

// The information-disclosure rule: a user who cannot see a thing must not
// be able to tell whether it exists. Three responses collapse to one.
const NOT_FOUND = { code: 'not_found', message: 'No matching record.' }

// does not exist        -> NOT_FOUND
// exists, not permitted -> NOT_FOUND  (or chapter 119's acknowledge policy)
// exists, deleted       -> NOT_FOUND

// And the timing version, which is easy to forget. A permission failure
// that returns in 4 ms and a genuine miss that returns in 40 ms are
// distinguishable, so the permission path does the same work.

For an internal assistant the timing channel is usually below the threat model — chapter 115 put existence disclosure to a colleague low on the loss list. It is named here because the same code becomes a multi-tenant product later, and the threshold changes without the code changing.

Aggregates

// The query is permitted. Every row is permitted. The result discloses
// something about rows that are not.
//
//   "What is the average order value across all customers?"
//
// A rep who can see 40 of 400 customers now knows something about 360.

// Two controls, neither complete.
//
// 1. Minimum cohort. An aggregate must cover at least N records the user
//    can see individually, or it is refused. Atlas uses 5.
if (visibleToUser < MIN_COHORT) return refuse('not enough visible records to aggregate')

// 2. Scope the aggregate to the permitted set and SAY SO. "Across the
//    12 accounts you can see, the average is $4,100" is honest and
//    useful; "the average is $3,800" is a disclosure dressed as a fact.

The second is the one that matters in practice, and it doubles as a correctness fix: an unscoped aggregate is also wrong for the asker, because it describes a population they are not working with.

The test

// Two users, every path, on every build. It has caught three regressions
// in Atlas: a cache key that lost its prefix in a refactor, a session
// loader whose ownership check moved behind a feature flag, and a trace
// exporter added for debugging that wrote unredacted fixtures.
test('no path crosses users', async () => {
  const secret = await seedRestricted({ group: 'finance', marker: 'MARKER-7731' })

  for (const path of [askQuestion, recallMemory, loadSession, exportTrace, runSubagent]) {
    const asFinance = await path(finance, secret.query)
    expect(JSON.stringify(asFinance)).toContain('MARKER-7731')   // the control case

    const asRep = await path(rep, secret.query)
    expect(JSON.stringify(asRep)).not.toContain('MARKER-7731')
  }
})

The control assertion in the middle is what stops this test rotting into a tautology. Without it, a change that breaks the path entirely makes the test pass.

Key points

Eight paths cross users without going through a query: answer caches, retrieval caches, unfiltered memory recall, shared session state, saved traces, error messages, subagents, and aggregates.
Make scoped keys the only constructible kind, with a branded type the cache accepts and nothing else.
Put a hash of the group list in the key. Narrowing a user's access then makes stale entries unreachable with no invalidation logic.
Atlas's cache hit rate fell 23% to 11% when keys became user-scoped. That is the price.
Redact traces for sharing and synthesise them for fixtures. The complaint-to-eval-case path runs through that function.
Collapse "does not exist", "not permitted" and "deleted" to one response, and make the timing match.
Aggregates disclose without quoting anything. Require a minimum visible cohort and scope the aggregate to what the user can see — which is also the correct answer for them.
Test every path with two users, and include a control assertion so a broken path cannot pass.

Next: output filtering, and its limits — the last thing between the model and the user.