Chapter 8 · Foundations

Setting up the project

The repository layout, model access without leaking a key, and the three decisions in the first commit that are irritating to change six months later.

10 min read

This is the first chapter with a working repository at the end of it. It assumes the slice from chapter 7 and the stack named in chapter 5: TypeScript on Node, Postgres with pgvector, and a hosted model behind one configuration module.

Three of the decisions below are hard to undo. They are marked.

Layout

atlas/
  src/
    config.ts          # 🔒 every vendor choice, in one file
    ingest/            # offline half: fetch, parse, chunk, embed
      sources/         # one module per source, same interface
      chunk.ts
      embed.ts
      run.ts
    retrieve/          # online half: search, filter, rank
      search.ts
      scope.ts         # 🔒 permissions, in the query
    generate/
      prompt.ts        # prompts as files, not string literals
      answer.ts
    db/
      migrations/      # 🔒 plain SQL, numbered, forward-only
      client.ts
  evals/
    questions.yaml     # the 34 questions from chapter 7
    run.ts
  scripts/

The ingest and retrieve split mirrors the offline and online halves from chapter 6, and it is worth enforcing with a lint rule rather than a convention: nothing in retrieve/ may import from ingest/. The first time somebody parses a document on the request path, that import is how it happens.

// .eslintrc — one rule that prevents a whole class of latency bug.
{
  "rules": {
    "no-restricted-imports": ["error", {
      "patterns": [{
        "group": ["**/ingest/**"],
        "message": "The request path must not import ingestion. See chapter 6."
      }]
    }]
  }
}

Decision 1: all vendor choices in one module

🔒 Hard to undo if you skip it, trivial if you do it now.

The model, the embedding model and the vector store will all change. The embedding model is the expensive one — changing it invalidates every vector you have stored — so the goal is not to make changes free, but to make the blast radius visible.

// src/config.ts — the only file that names a vendor.
export const config = {
  generation: {
    provider: 'anthropic',
    model: process.env.ATLAS_MODEL ?? 'claude-sonnet-5-5',
    maxOutputTokens: 1500,
  },
  embedding: {
    // ⚠️ Changing either of these requires a full re-embed of every chunk.
    // The dimension is written into the database schema.
    provider: 'openai',
    model: 'text-embedding-3-small',
    dimensions: 1536,
  },
  retrieval: {
    candidates: 50,   // fetched before reranking
    window: 12,       // sent to the model
  },
} as const

export type Config = typeof config

Writing the embedding dimension here, as a constant that the migration also uses, is the small discipline that catches the worst version of this mistake: switching embedding models, forgetting the dimension changed, and getting a Postgres error at insert time on a 40,000-chunk re-index that has been running for twenty minutes.

Decision 2: the schema

🔒 Genuinely hard to change later, because it holds the permission metadata.

-- db/migrations/001_chunks.sql
create extension if not exists vector;

create table documents (
  id            text primary key,
  source        text not null,          -- 'policy_docs', 'customer_contracts', ...
  source_ref    text not null,          -- the id in the system of record
  title         text not null,
  updated_at    timestamptz not null,
  -- Permission metadata. Denormalised on purpose: the retrieval query must not
  -- join to another system to decide what somebody may see. See chapter 13.
  classification text not null,         -- 'public_internal' | 'account' | 'financial'
  account_id    text,                   -- null unless classification = 'account'
  deleted_at    timestamptz
);

create table chunks (
  id            bigserial primary key,
  document_id   text not null references documents(id) on delete cascade,
  ordinal       int not null,            -- position within the document
  heading_path  text[] not null default '{}',  -- ['Returns', 'Damaged goods']
  content       text not null,
  tokens        int not null,
  embedding     vector(1536) not null,
  unique (document_id, ordinal)
);

-- The index that makes permission filtering cheap rather than a sequential scan.
create index chunks_doc on chunks(document_id);
create index documents_scope on documents(classification, account_id)
  where deleted_at is null;

Three things in that schema are there for reasons that only become obvious later.

heading_path is the document structure, carried through the chunker. It is how a chunk that says "within 30 days of delivery" can be rendered to the model as Returns → Damaged goods → within 30 days of delivery. Chapter 32 is about not losing it; adding it afterwards means re-ingesting.

classification and account_id live on documents, not chunks. Permissions are a property of the document, and duplicating them per chunk means 45,000 rows to update when one contract changes hands. The join is to a table in the same database, which is the one join the request path can afford.

deleted_at rather than a delete. Chapter 35 is entirely about the chunk that outlives its document, and a soft delete with a partial index is the cheapest correct answer.

Decision 3: prompts are files

🔒 Not hard to change, but it changes how you work.

A prompt is an engineering artifact — chapter 53 makes that argument properly. The practical consequence is that prompts live in files, get reviewed in pull requests, and are versioned alongside the code that uses them.

// src/generate/prompt.ts
import fs from 'node:fs'
import path from 'node:path'

const DIR = path.join(import.meta.dirname, 'prompts')

/** Loaded once at startup. A prompt change is a deploy, not a runtime edit. */
const cache = new Map<string, string>()

export function prompt(name: string, vars: Record<string, string>): string {
  if (!cache.has(name)) {
    cache.set(name, fs.readFileSync(path.join(DIR, `${name}.md`), 'utf8'))
  }
  // Deliberately strict: an unfilled placeholder is a bug, not an empty string.
  return cache.get(name)!.replace(/\{\{(\w+)\}\}/g, (_, key) => {
    if (!(key in vars)) throw new Error(`prompt ${name}: missing variable ${key}`)
    return vars[key]
  })
}

Throwing on a missing variable rather than substituting an empty string is worth the three lines. A prompt with a silently empty {{sources}} block does not error — it produces a fluent, completely unsourced answer, and that is the hardest failure in this whole system to notice.

Model access without leaking a key

The rules are short and there is no interesting version of them.

# .env.local — gitignored, never committed, never printed in a log line.
ANTHROPIC_API_KEY=...
OPENAI_API_KEY=...
DATABASE_URL=postgres://localhost:5432/atlas

// src/config.ts, continued. Fail at startup, not at first request.
function required(name: string): string {
  const value = process.env[name]
  if (!value) throw new Error(`Missing required environment variable: ${name}`)
  return value
}

export const secrets = {
  anthropicKey: required('ANTHROPIC_API_KEY'),
  openaiKey: required('OPENAI_API_KEY'),
  databaseUrl: required('DATABASE_URL'),
}

Two things that are easy to get wrong:

Never log a request body that contains a prompt. Prompts carry retrieved chunks, and retrieved chunks carry whatever was in the documents. A debug log of full requests is an unaudited copy of the corpus in your log aggregator, exported to wherever that is hosted.
Scope keys per environment. Development, staging and production get separate keys, so a leaked development key does not read production data and a spend spike is attributable.

Proving it works

The setup is done when this passes:

npm run migrate                 # applies db/migrations in order
npm run check                   # tsc --noEmit, then eslint
npm run smoke                   # inserts one chunk, searches for it, deletes it

// scripts/smoke.ts — the smallest end-to-end proof.
const text = 'Damaged goods may be returned within 30 days of delivery.'
const [vector] = await embed([text])

await db.query(
  `insert into documents (id, source, source_ref, title, updated_at, classification)
   values ('smoke', 'policy_docs', 'smoke', 'Smoke test', now(), 'public_internal')`,
)
await db.query(
  `insert into chunks (document_id, ordinal, content, tokens, embedding)
   values ('smoke', 0, $1, $2, $3)`,
  [text, 12, toVector(vector)],
)

const [query] = await embed(['how long do I have to return damaged items'])
const { rows } = await db.query(
  `select content, 1 - (embedding <=> $1) as score
     from chunks order by embedding <=> $1 limit 1`,
  [toVector(query)],
)

console.log(rows[0].score > 0.3 ? '✅ smoke passed' : '❌ similarity too low')
await db.query(`delete from documents where id = 'smoke'`)

That script is worth keeping. It is the test that tells you whether a mystifying failure is in your retrieval logic or in the connection to the embedding provider, and it answers in two seconds.

The failure mode: a schema that cannot express a permission

The version of this chapter that goes wrong looks right for about three weeks. You build documents without classification and account_id, because the first thirty questions are all policy questions and policy documents are visible to everyone. Retrieval works. Answers are good.

Then contracts are added, and contracts are per-account. Now the filter needs data that is not in the table, so it is fetched from the document store at request time — one extra call, which is fine — and then from the CRM, to resolve which accounts the asker owns. The request path now depends on two external systems to decide what to show, both of which can be slow and one of which can be down.

The correct version is the one in the migration above: the data needed to make a permission decision lives in the same database as the chunks, and is written during ingestion. Chapter 13 is where that gets designed rather than asserted.

Key points

Enforce the ingest/retrieve split with a lint rule. Convention does not survive a deadline.
Every vendor choice in one config module, with the embedding dimension as a shared constant.
Permission metadata belongs on the document table, written at ingestion, in the same database as the chunks.
heading_path and deleted_at are cheap now and require a re-ingest later.
Prompts are files. Throw on a missing variable: a silently empty sources block produces a fluent, unsourced answer.
Keep the two-second smoke test. It separates your bugs from the provider's.

Next: a hello-world agent — one question, one document, an answer with a citation, and the end-to-end path working before anything is scaled up.