Chapter 50 · Retrieval

Tuning against the test set, and knowing when to stop

Eleven parameters, a search order that respects their interactions, and the three stopping rules. The last chapter of the retrieval section, and the one about not over-engineering it.

12 min read

Section 4 has introduced eleven tunable parameters. This chapter searches them in a sensible order and then argues for stopping, which is the harder half.

It assumes the harness and a test set of real questions.

The parameters

Parameter	Range	Interacts with	Chapter
`ef_search`	20–320	Everything downstream	40
Candidate limit	25–100	Rerank keep, latency	44
Score floor	0.2–0.5	Abstention rate	41
Per-document cap	1–5	Window size	41
RRF `k`	20–120	Fusion balance	43
Vector weight	0.3–0.7	RRF `k`	43
Rerank keep	3–12	Tokens, generation quality	44
Rerank min score	0–0.3	Abstention rate	44
MMR lambda	0.5–1.0	Per-document cap	44
Glossary terms	—	Keyword and vector sides	46
Source boost	1.0–1.5	Precision	45

Eleven parameters is a search space of millions if gridded naively. It is tractable because they are mostly independent and they have a clear priority order.

The search order

The order matters because of one-way dependencies. Tuning the rerank keep before fixing the candidate limit optimises against a shallow candidate set, which is chapter 44's failure mode. Tuning the floor before recall is settled calibrates against a distribution that is about to change.

// scripts/tune.ts — coordinate descent, one group at a time, in order.
// Not a grid: sweep one parameter, fix the winner, move on, and loop twice.
const GROUPS = [
  { name: 'correctness', params: ['efSearch', 'candidateLimit'] },
  { name: 'recall',      params: ['rrfK', 'vectorWeight'] },
  { name: 'precision',   params: ['rerankKeep', 'perDocumentCap', 'mmrLambda'] },
  { name: 'abstention',  params: ['scoreFloor', 'rerankMinScore'] },
]

export async function tune(cases: RetrievalCase[], base: RetrievalConfig) {
  let best = base
  for (let pass = 0; pass < 2; pass++) {       // two passes catches interactions
    for (const group of GROUPS) {
      for (const param of group.params) {
        best = await sweepOne(cases, best, param, RANGES[param], OBJECTIVE[group.name])
      }
    }
  }
  return best
}

Two passes rather than one is the concession to interaction. A third pass has never changed anything on this corpus, and if it does, that is a sign the parameters are more coupled than assumed.

Different objectives per group

Optimising everything for one number is how you get a configuration that scores well and behaves badly.

const OBJECTIVE = {
  // Recall only, at the candidate depth. Precision here is meaningless —
  // the reranker's job is to fix it.
  correctness: (r: EvalReport) => r.summary.recallAtCandidateDepth,
  recall: (r: EvalReport) => r.summary.recallAtCandidateDepth,

  // Precision subject to a recall constraint, which is the honest version:
  // a precision gain that costs recall is not a gain.
  precision: (r: EvalReport) =>
    r.summary.recall >= 0.95 ? r.summary.precision : -1,

  // Abstention is a trade between two error types, and they are not equal.
  // A false abstention is an annoyance; a confident wrong answer is damage.
  abstention: (r: EvalReport) =>
    r.summary.correctAbstentionRate * 2 - r.summary.falseAbstentionRate,
} as const

The -1 in the precision objective is a hard constraint rather than a penalty, and it is deliberate: it makes any configuration that drops recall below 0.95 strictly worse than any that does not, regardless of how good its precision is.

Atlas's results

120 cases, 84 train and 36 held out.

Parameter	Start	Tuned	Δ recall	Δ precision
`ef_search`	40	100	+0.03	0
Candidate limit	50	50	0	0
RRF `k`	60	60	0	0
Vector weight	0.5	0.5	0	0
Rerank keep	6	6	0	0
Per-document cap	3	2	−0.01	+0.04
MMR lambda	0.7	—	0	0
Score floor	0.33	0.30	+0.02	−0.01
Rerank min score	0.1	0.1	0	0

Two parameters moved meaningfully. Seven did not move at all.

train (84):     recall 0.94 -> 0.97   precision 0.38 -> 0.42
holdout (36):   recall 0.92 -> 0.94   precision 0.37 -> 0.40

The holdout improving is what makes this real rather than fitted. A train gain of +0.03 with a holdout gain of +0.02 is a genuine improvement; a train gain of +0.03 with a flat holdout would have been noise with a configuration change attached.

Three readings worth taking from a table like that:

Most parameters do not matter. The defaults were chosen by measurement in their own chapters, and re-tuning them jointly found nothing. That is a good outcome, not a wasted exercise — it means the per-chapter decisions were sound.

ef_search was the largest single gain, and it is a database setting most teams never touch. Chapter 40 predicted this.

The per-document cap traded recall for precision and the trade was taken because the recall loss was one case and the precision gain was material. That is a judgement, and it should be recorded as one.

The three stopping rules

1. The holdout stopped moving

// Two consecutive groups with no holdout improvement: stop.
if (holdoutGain < 0.01 && previousHoldoutGain < 0.01) {
  logger.info('tune.converged', { passes, finalConfig: best })
  return best
}

2. The remaining failures are not retrieval failures

holdout failures after tuning (36 cases, 2 failing):

ctr-003  not_ingested   scanned contract, excluded (chapter 20)
neg-002  ranked_out     negation: "may not be returned" ranked below
                        "may be returned" (chapter 38)

Neither is fixable by a parameter. The first is a data decision and the second is a known limitation of embeddings that generation has to handle. Continuing to tune against them produces a configuration contorted to catch two cases.

3. The gain is smaller than the variance

// Measure the noise before chasing a gain smaller than it.
export async function noiseBand(cases: RetrievalCase[], config: RetrievalConfig, runs = 5) {
  const scores = []
  for (let i = 0; i < runs; i++) {
    // Resample the cases: the variance that matters is over which questions
    // you happen to have, not over the retrieval itself, which is deterministic.
    scores.push((await runRetrievalEval(bootstrap(cases), config)).summary.recall)
  }
  return { mean: mean(scores), sd: standardDeviation(scores) }
}
// Atlas, 120 cases: mean 0.96, sd 0.018.
// ⚠️ So anything under ~0.036 (2 sd) is not distinguishable from noise.

A +0.01 improvement on 120 cases is one question. Shipping a more complex configuration for one question is a bad trade, and the rule that follows is worth stating plainly:

Prefer the simpler configuration unless the gain exceeds two standard deviations of the bootstrap. Complexity you cannot justify statistically is complexity you will not be able to justify to the person maintaining it.

What to do instead of tuning further

When the stopping rules fire, the remaining gains are elsewhere. In rough order of value:

Action	Why it beats more tuning
Fix the data	Six of Atlas's fourteen no-result cases were scanned contracts. OCR beats any parameter.
Add glossary terms	Five more were jargon. A data file, twenty minutes.
Grow the test set	A wider set reveals failures tuning cannot see.
Improve generation	Recall 0.96 with a model that misreads the passage is still a wrong answer.
Ship it	0.94 on held-out real questions is a working system.

The last row is the honest one. Atlas's target was 70% coverage; retrieval at 0.94 recall is comfortably ahead of what generation and adoption will deliver, and further retrieval work is optimising the component that is no longer the constraint.

Recording what was decided

// Config with provenance. Every number says when it was measured and what
// it scored, so the next person is not guessing.
export const RETRIEVAL_CONFIG = {
  efSearch: 100,          // tuned 2026-11-03, +0.03 recall over 40
  candidateLimit: 50,     // chapter 44, knee; 100 gave no gain
  rrfK: 60,               // conventional; sweep 20-120 found nothing
  vectorWeight: 0.5,      // sweep 0.3-0.7 moved recall < 0.01
  rerankKeep: 6,          // chapter 44; 3 lost recall, 12 doubled tokens
  perDocumentCap: 2,      // tuned 2026-11-03, -0.01 recall / +0.04 precision
  scoreFloor: 0.30,       // calibrated 2026-11-03; recalibrate as corpus grows
  rerankMinScore: 0.1,

  measuredOn: '2026-11-03',
  testSetSize: 120,
  holdoutRecall: 0.94,
  holdoutPrecision: 0.40,
  noiseSd: 0.018,          // ⚠️ do not act on gains below ~0.036
} as const

Recording noiseSd in the config is unusual and it is the line that saves the most future time. It tells whoever next proposes a change exactly how large that change has to be before it means anything.

Section 4 is done

	Start of section	End
recall@12 (written set)	0.89	0.99
recall@6 (held-out real)	—	0.94
precision@6	0.11	0.40
Tokens per question	5,040	2,520
p50 retrieval latency	9 ms	214 ms
p50 total latency	2,474 ms	1,774 ms

Precision nearly quadrupled, tokens halved, and total latency fell despite retrieval getting twenty times slower — because reranking saved more generation time than it cost.

Section 5 is the model layer, and it starts where this one ends: six chunks, 2,520 tokens, 0.94 recall. Everything from here is about what a model does with them.

The failure mode: the tuning that never ends

The parameters are there, the harness runs in seconds, and each sweep finds another hundredth. It is absorbing work with visible progress and it is the most common way a project of this kind spends a month.

Three symptoms, each with a specific counter-question.

Symptom	Ask
Train improving, holdout flat	Is this fitting or improving?
Gains under two standard deviations	Is this distinguishable from noise?
Parameters above seven in the config	Can the next person reason about this?

// A budget, set before starting, because the stopping rules are easier to
// follow when the decision was made in advance.
const TUNING_BUDGET = {
  hours: 4,
  minHoldoutGain: 0.03,    // two sd
  maxParameters: 8,
} as const

The structural point, and the one worth carrying into section 5:

Retrieval is a component with a measurable ceiling, and you will hit it. When the holdout stops moving, the constraint has moved somewhere else — and continuing to work here is optimising a part of the system that is no longer the problem.

Key points

Search in order: correctness, recall, precision, abstention, latency. The dependencies are one-way.
Coordinate descent with two passes, not a grid. A third pass finding something means the parameters are more coupled than assumed.
Use a different objective per group, with recall as a hard constraint on the precision objective.
Expect most parameters not to move. Seven of nine did not, which validates the per-chapter decisions.
ef_search was the largest single gain and is a database setting most teams never touch.
Three stopping rules: the holdout stops moving, the remaining failures are not retrieval failures, or the gain is under two standard deviations.
Measure the noise band by bootstrapping the cases, and record it in the config.
When tuning stops paying, fix the data, add glossary terms, grow the test set, or ship.
Set a tuning budget in advance. The stopping rules are much easier to follow when the decision was already made.

Next, and the start of section 5: choosing a model, and the criteria that actually matter — beginning from six chunks and 2,520 tokens.