Chapter 17 · The data layer

Residency, retention and what your contracts already promise

The obligations you have already signed up to, and the four ways an index quietly breaks them. Read the contracts before the architecture is fixed.

11 min read

The previous chapter decided what personal data may enter the system. This chapter is about constraints on data that is perfectly fine to hold — where it may be held, how long, and who may process it.

These constraints already exist. Nobody is going to tell you about them, because the people who negotiated them are not in the project, and the systems that currently comply with them are not the system you are building. The job here is archaeology, and it is worth two days.

As in the previous chapter: this is engineering guidance on what the obligations mean architecturally, not legal advice.

The four ways an index breaks a promise

Each is a category of promise your company has probably made somewhere, and each is broken by a default architecture rather than by a mistake.

A new copy in a new place

Your document store may be in a specific region because a customer contract requires it. An index in a different region is a copy outside that boundary, and the fact that it holds extracted chunks rather than original files does not change the analysis.

-- Record it per source, so a region question has an answer without a meeting.
alter table documents add column residency_region text not null default 'unknown';
alter table documents add constraint documents_residency_known
  check (residency_region <> 'unknown');

Defaulting to 'unknown' and then forbidding it with a constraint is a deliberate pattern: it makes the omission fail loudly at insert time rather than producing a column of nulls nobody notices.

A new processor

This is the one most often missed, and it is the biggest change in the data flow.

Every retrieved chunk is sent to a model provider. If your contracts name sub-processors, or promise that customer data is not disclosed to third parties, or require notification before new processors are engaged, then adding a model provider is a contractual event rather than an infrastructure choice.

The questions to answer before building, in order:

Does the provider train on your inputs by default? For business and enterprise tiers, generally no — but the answer depends on the tier and the agreement, and it is a term, not a reputation.
What is the provider's retention period for request data, and can it be set to zero?
Where is inference performed, and is a specific region available?
Is the provider on your company's approved processor list, and if not, what does adding one involve?

// Make the data-flow boundary explicit in code. Everything in this function
// crosses an organisational boundary, and the comment is for the reviewer who
// is not thinking about compliance.
/**
 * ⚠️ DATA EGRESS. Content passed here leaves the company's infrastructure and
 * is processed by the provider named in config.generation.provider.
 * Residency and retention are governed by the agreement in docs/processors.md.
 * Do not add a call site without reading it.
 */
export async function callModel(prompt: string, options: CallOptions) { /* ... */ }

A comment is not a control. It is, however, the thing a reviewer reads at the moment they are about to introduce a second egress path, and that is worth more than its cost.

A retention clock nobody set

Retention schedules say how long a record is kept. An index inherits nothing: it keeps chunks until something deletes them.

The common shape of the problem is that a source system expires records after a defined period — say support tickets after two years — and the index has no such rule, so three-year-old ticket text remains retrievable after the source has correctly disposed of it.

-- Retention as data, per source, enforced by a job rather than remembered.
create table source_retention (
  source        text primary key,
  retention_days int,                 -- null means "as long as the source keeps it"
  basis         text not null         -- why this number. A reference, not a guess.
);

insert into source_retention values
  ('policy_docs',       null,  'Current version only; superseded versions removed at ingest'),
  ('customer_contracts', null, 'Retained while the account is active; follows source'),
  ('support_tickets',    730,  'Matches the ticket system retention schedule');

// The job. Runs nightly, reports what it removed, and alerts if it removes
// nothing for a source that should be expiring something.
export async function enforceRetention() {
  const rules = await db.query(
    `select source, retention_days from source_retention where retention_days is not null`,
  )

  for (const { source, retention_days } of rules.rows) {
    const { rowCount } = await db.query(
      `update documents
          set deleted_at = now()
        where source = $1
          and deleted_at is null
          and updated_at < now() - make_interval(days => $2)`,
      [source, retention_days],
    )
    logger.info('retention.enforced', { source, expired: rowCount })
  }
}

The basis column is the part that makes this maintainable. A retention number with no recorded reason gets changed by whoever next finds it inconvenient.

Derived data that outlives its source

Deleting a document and leaving its chunks is chapter 35's subject. The broader version of the problem is everything else derived from a document:

Derived artifact	Holds source content?	Where it is handled
Chunks and embeddings	Yes, verbatim	Cascade on delete, chapter 35
Conversation logs	Yes, in the answer text	Chapter 124
Agent memory	Yes, summarised	Chapter 96
Eval fixtures	Yes, and committed to git	Here. This one is permanent.
Traces and request logs	Yes, if context is logged	Previous chapter
Response caches	Yes	Chapter 132

The eval fixture row deserves its emphasis. A test set built from real documents, committed to a git repository, is not deletable in any practical sense — git history is designed to prevent exactly that. The fix is a rule adopted before the first fixture is written:

# evals/questions.yaml — reference content, never copy it.
- id: ret-001
  question: "How long do customers have to return damaged goods?"
  # ✅ A reference and an assertion about the answer.
  expect:
    must_contain: ["30 days"]
    must_cite: ["Returns and Credits Policy"]
  # ❌ Never: a verbatim excerpt of the source document, which would make this
  #    file an uncontrolled, permanent copy of corpus content.

Assert on short factual values and document titles, not on passages. It is a small constraint that also makes the eval set more robust to rewording.

Reading the contracts

The archaeology, in the order that gets to an answer fastest.

Start with the customer-facing templates, not individual contracts. Most companies have two or three standard agreements. Read the data-protection and confidentiality sections of each. That covers most of the estate in an hour.

Then find the negotiated exceptions. Large customers amend the template, and the amendments are where the region-specific and sub-processor clauses live. Sales operations or legal can usually name the handful that were negotiated.

Then the inbound side. Your own vendor agreements and any certification the company holds impose obligations too, and a certification scope statement is a useful summary of what the company has told auditors about where data lives.

The output is one table:

Source	Residency	Retention	Processor restriction	Reference
Policy documents	None	None	None	Internal material
Product specifications	None	None	Three supplier NDAs restrict onward disclosure	NDA register
Customer contracts	9 accounts require in-region	Follows account lifecycle	4 accounts require processor notification	Amendment register
Support tickets	None	730 days	None	Ticket system policy

Nine accounts and four accounts. Those two numbers determine whether the architecture needs per-document residency routing, and they are findable in an afternoon.

When a constraint conflicts with the design

It happens. Nine accounts requiring in-region processing, and your chosen model provider has no presence in that region.

Four options, in order of preference:

Exclude those documents from the index. Nine accounts out of 4,000 is a small coverage loss and a large simplification. Answer those questions the way they are answered today.
Route by classification. Those documents are indexed in-region and served by an in-region deployment. Correct, and roughly doubles the operational surface.
Seek a contract amendment. Slow, sometimes easy, and worth starting in parallel with option 1.
Proceed and document the gap. Only with explicit, informed sign-off from someone with authority to accept it, in writing.

Option 1 is right far more often than it is chosen, because excluding data feels like failure. It is not: a system that covers 99% of accounts and is unambiguously compliant ships, and one that covers 100% and needs a legal review does not.

The failure mode: the retroactive discovery

The way this goes wrong is a sequence that looks fine at every step.

The index is built. It works well, and it is in the default region of whichever platform the team already uses. Three months later a customer sends a routine security questionnaire asking where their data is processed and which sub-processors are engaged.

Now the question is not "where should the index be?" but "where has this customer's data been for three months, and who do we have to tell?" The engineering fix is a migration. The hard part is the disclosure, and it is hard in proportion to how long the gap existed.

The whole defence is the two days spent on the table above, before the first ingest. There is no clever version of this. ⚠️ A residency decision made by default is still a decision, and it is the only one in this chapter that cannot be quietly corrected later.

Key points

An index breaks promises in four ways: a new copy, a new processor, an unset retention clock, and derived data outliving its source.
The model provider is a new processor. Check training, retention and region as contract terms, not reputations.
Default residency to 'unknown' and forbid it with a constraint, so the omission fails at insert time.
Record retention per source with a written basis. A number with no reason gets changed by whoever finds it inconvenient.
Eval fixtures committed to git are permanent copies. Assert on short values and titles, never on passages.
Read the standard templates first, then the negotiated amendments. Two days produces the table that decides the architecture.
Excluding a handful of constrained documents is usually the right answer and is rarely chosen.

Next: mapping sources to questions — which data answers which asks, and the questions that turn out to have no source at all.