Chapter 32 · Ingestion

Keeping document structure through the chunker

A chunk that does not know where it came from is a fragment. The heading path, what it costs, what it buys at retrieval and at generation, and the three places it gets lost.

11 min read

Every chapter in this section has carried headingPath through its types without quite justifying it. This chapter is the justification, and the implementation.

The claim is small and the effect is not: a chunk that does not know where it came from is a fragment, and fragments retrieve badly, read badly and cite badly.

What a chunk looks like without it

within 30 days of delivery, provided the damage is reported within 48 hours
of receipt. Items returned after this period are assessed individually.

Within 30 days of what? Returned from where? This passage is unambiguous in its document, under Returns → Damaged goods, and meaningless out of it. Three distinct things break.

Stage	What breaks
Embedding	The words "returns" and "damaged" are absent, so the chunk does not match the question
Generation	The model sees a rule with no subject and must guess what it governs
Citation	"Returns and Credits Policy" is a 40-page document; the reader cannot check the claim

With the path, the same chunk becomes:

Returns and Credits Policy > Returns > Damaged goods

within 30 days of delivery, provided the damage is reported within 48 hours
of receipt. Items returned after this period are assessed individually.

Building the path

A stack, maintained as blocks stream past. The subtlety is truncation on depth changes.

// src/ingest/chunk/heading-path.ts
export class HeadingStack {
  private readonly stack: string[] = []

  push(depth: number, text: string): void {
    // A depth-2 heading clears everything at depth 2 and below. Without the
    // truncation, 'Returns > Damaged goods' persists into the next section
    // and every chunk after it is labelled with a section it is not in.
    this.stack.length = Math.max(0, depth - 1)
    this.stack[depth - 1] = text.trim()
  }

  /** Dense: skipped levels are omitted rather than left as holes. A document
   *  that jumps h1 -> h3 should not produce ['Scope', undefined, 'Exclusions']. */
  path(): string[] {
    return this.stack.filter((s): s is string => Boolean(s))
  }
}

export function chunkWithPath(blocks: Block[], target: number): Chunk[] {
  const headings = new HeadingStack()
  const chunks: Chunk[] = []
  let buffer: Block[] = []

  const flush = () => {
    if (buffer.length === 0) return
    chunks.push({
      ordinal: chunks.length,
      content: renderBlocks(buffer),
      // ⚠️ Captured at flush, not at push. A chunk spanning a heading boundary
      // must carry the path it STARTED in, or every boundary chunk is mislabelled.
      headingPath: headings.path(),
      kind: buffer.every((b) => b.type === 'table') ? 'table' : 'prose',
      tokens: countTokens(renderBlocks(buffer)),
    })
    buffer = []
  }

  for (const block of blocks) {
    if (block.type === 'heading') {
      flush()                       // a heading always starts a new chunk
      headings.push(block.depth, block.text)
      continue
    }
    buffer.push(block)
    if (countTokens(renderBlocks(buffer)) >= target) flush()
  }
  flush()
  return chunks
}

Flushing on every heading is what keeps the invariant simple: a chunk never spans a heading, so its path is unambiguous. It costs some short chunks under short sections, and the minimum-length invariant from chapter 30 catches the pathological ones.

Using it in three places, differently

The path is not one string used three times. Each consumer wants a different rendering, which is why content and embedText are separate fields in the pipeline's types.

// 1. Embedding: prefix the path. This is the measurable retrieval win.
embedText: `${[doc.title, ...chunk.headingPath].join(' > ')}\n\n${chunk.content}`

// 2. Generation: render as a labelled source, so the model can cite precisely.
`[${n}] ${doc.title} — ${chunk.headingPath.join(' > ')}\n${chunk.content}`

// 3. Display: the path is a breadcrumb the reader can follow into the document.
{ title: doc.title, breadcrumb: chunk.headingPath, href: `${doc.url}#${chunk.anchor}` }

Including the document title in the embedded text and not in the displayed content is deliberate. The title carries topic words that help matching, and repeating it in the body of a rendered source wastes tokens on something already in the source label.

On Atlas's policy corpus, prefixing the path moved recall@12 from 0.84 to 0.91 — the same magnitude as the entire chunk-size tuning in chapter 31, for about fifteen lines of code. It is the highest-return change in this section.

Three places it gets lost

Parsing produced no headings

For a scanned PDF, headings are inferred from font clustering and the inference sometimes fails. The result is a document of 200 paragraph blocks and no structure at all.

// Fall back to a synthetic path rather than to nothing. A page number is a
// weak referent and it is much better than an empty array for citation.
if (doc.blocks.every((b) => b.type !== 'heading')) {
  doc.parseWarnings.push('no headings detected — using page anchors')
  return chunkWithPageAnchors(doc)    // ['Page 14']
}

A chunk is split mid-section

Section 4.2 is 3,000 tokens and becomes five chunks. All five carry the same path, which is correct and leaves them indistinguishable in a citation list.

// Number the parts, so five identical citations become five distinct ones.
const parts = splitOversized(section, target)
parts.forEach((part, i) => {
  part.headingPath = parts.length > 1
    ? [...basePath, `part ${i + 1} of ${parts.length}`]
    : basePath
})

The table lost its caption

A table block chunked alone retains its headers and loses the sentence above it that said what the table is of.

// A table chunk carries the preceding paragraph as a lead-in. One paragraph,
// capped, because the alternative is a table that is mostly prose.
function tableChunk(table: TableBlock, preceding: Block | undefined, path: string[]): Chunk {
  const lead = preceding?.type === 'paragraph' && preceding.text.length < 300
    ? `${preceding.text}\n\n` : ''
  return {
    content: `${lead}${table.caption ? `${table.caption}\n\n` : ''}${renderTable(table)}`,
    headingPath: path,
    kind: 'table',
    // ...
  }
}

Storing and querying it

-- heading_path is text[], not a joined string. The array is queryable.
create index chunks_heading_path on chunks using gin (heading_path);

-- Which makes two useful things cheap. First: resolving the test set's
-- document+heading references, from chapter 18.
select id from chunks
 where document_id = $1 and heading_path = $2;

-- Second: sibling expansion scoped to a section rather than to a window of
-- ordinals — more precise than the neighbour query in chapter 30.
select id, content from chunks
 where document_id = $1
   and heading_path[1:2] = $2[1:2]     -- same section, any subsection
 order by ordinal;

Section-scoped expansion is the better version of neighbour retrieval. An ordinal window of ±1 can cross a section boundary and pull in an unrelated chunk; a path prefix cannot.

A structural signal worth keeping

Once the path exists, it carries information beyond labelling. Depth correlates with specificity, and position correlates with importance.

// Mild, principled ranking adjustments. ⚠️ Apply after retrieval, measure the
// effect, and remove them if the test set does not improve — this is exactly
// the kind of tweak that feels right and does nothing.
export function structuralBoost(chunk: Retrieved): number {
  let boost = 1.0
  // Definitions and scope sections answer a disproportionate share of questions.
  if (/\b(definitions?|scope|overview|summary)\b/i.test(chunk.headingPath.join(' '))) {
    boost *= 1.08
  }
  // Appendices and annexes are usually supporting detail.
  if (/\b(appendix|annex|exhibit|schedule \d)\b/i.test(chunk.headingPath.join(' '))) {
    boost *= 0.92
  }
  return boost
}

The warning in that comment is the point of including it. Structural heuristics are easy to write and easy to convince yourself about; the test set from chapter 31 is how you find out whether this one earns its place on your corpus.

The failure mode: the path that is a different document's

Atlas's contracts are ingested with a shared parser. A contract whose first page is a cover sheet with a large title and no heading markup produces, through font-size clustering, a depth-1 heading of the customer's name.

Every chunk in that contract now carries ['Henderson Industrial Supply', 'Terms', ...]. Harmless — until two contracts for related entities are both indexed, and a question about Henderson retrieves chunks from the Henderson Pacific contract because the path matches strongly and the body text of a standard clause is identical across all 3,100 contracts.

The permission filter still holds; this is not a leak. It is a wrong answer about the wrong entity, with a citation that looks right.

// The guard: the path is document structure, and the document's identity
// comes from source metadata. They are different things and the first must
// not be allowed to impersonate the second.
export function validateHeadingPath(chunk: Chunk, doc: TransformedDocument): void {
  const first = chunk.headingPath[0]
  if (!first) return

  // A depth-1 heading that matches the document title is redundant, not wrong.
  if (normalise(first) === normalise(doc.title)) {
    chunk.headingPath = chunk.headingPath.slice(1)
    return
  }

  // ⚠️ A depth-1 heading that is an account name from the CRM is a cover
  // sheet misread as structure. Drop it — the account is already on the
  // document, where the permission filter reads it.
  if (isKnownAccountName(first) && first !== doc.accountName) {
    logger.warn('chunk.suspicious_heading', {
      documentId: doc.id, heading: first, documentAccount: doc.accountName,
    })
    chunk.headingPath = chunk.headingPath.slice(1)
  }
}

The general rule, which is the same one chapter 14 gave about classification:

Structure comes from the document. Identity comes from the source system. Never let inferred structure supply an identity.

Key points

A chunk without its heading path is a fragment, and it breaks embedding, generation and citation in three different ways.
Prefixing the path onto the embedded text moved recall@12 from 0.84 to 0.91 on Atlas's corpus — the same magnitude as all of the chunk-size tuning, for fifteen lines.
Truncate the heading stack on depth change, keep it dense, and capture the path at flush rather than at push.
Flush on every heading so a chunk never spans one. The path is then unambiguous.
Render the path differently for embedding, generation and display. They want different things.
It gets lost three ways: no headings parsed, a section split into parts, a table separated from its caption. All three have cheap fallbacks.
Store it as text[] with a GIN index. Section-prefix expansion beats an ordinal window.
Structural boosts are easy to invent and must be measured against the test set.
Structure comes from the document; identity comes from the source system. Never let inferred structure supply an identity.

Next: chunk metadata — everything else that travels with a chunk, and why the permission fields are not negotiable.