Chapter 15 · The data layer

Structured against unstructured

A table and a paragraph are not two formats of the same thing. What breaks when you chunk a spreadsheet, and the three strategies for data that is both.

11 min read

Chapter 12 split sources by shape and routed documents to the index and databases to live queries. This chapter is about the awkward middle: the sources that are structured but are not a database, and the documents that contain tables.

That middle is larger than it sounds. Atlas's inventory found a pricing workbook that answers four questions, and 1,900 product specifications in which the useful content is almost entirely tabular.

What makes structure different

A paragraph means something as a continuous span of text. A table row means something only in relation to its headers and often to other rows.

Consider one row from the pricing workbook:

Region	Tier	Discount	Min order	Effective
North	3	12%	$25,000	2026-01-01

Chunked as text, that becomes North | 3 | 12% | $25,000 | 2026-01-01. Four problems follow immediately, and only the first is obvious.

The headers are gone. Is 3 a tier, a quantity, or a number of days? Is 12% a discount or a tax?
Comparison is impossible. "Which region has the best tier-3 discount" requires all the rows at once, and retrieval returns a handful.
Similarity is near-random. Every row in the table is textually almost identical. The embedding of the North row and the South row differ in the tokens that carry least semantic weight, which is the same problem as embedding database rows.
Aggregation invents. Asked for the average discount, a model given six of forty rows will produce a confident average of six rows and will not mention that it had six.

The fourth is the dangerous one, because the output is a plausible number with no indication of its basis.

The decision

The test is the question, not the file format. "What is the tier-3 discount for the north region?" is an individual fact. "Which regions have a minimum order above $20,000?" is across rows. A workbook that gets both kinds of question needs both treatments.

Strategy 1: serialise with headers

When the questions are lookups, this is cheap and works well. The key is that each row is self-describing.

// Each row becomes a chunk that carries its own headers. Verbose on purpose:
// the chunk has to make sense alone, because that is how it will be read.
export function serialiseRow(
  headers: string[],
  row: string[],
  context: { sheet: string; table?: string },
): string {
  const pairs = headers
    .map((h, i) => (row[i] ? `${h}: ${row[i]}` : null))
    .filter(Boolean)
    .join('\n')

  return `${context.table ?? context.sheet}\n\n${pairs}`
}

Which produces:

Regional discount tiers

Region: North
Tier: 3
Discount: 12%
Min order: $25,000
Effective: 2026-01-01

That chunk retrieves well for "north region tier 3 discount", because the question's words now appear in it. The plain pipe-delimited version does not, and the difference in retrieval quality is large — this single change is often the difference between a workbook being useless and being the best source in the corpus.

⚠️ One constraint: this only works while the row count is modest. Forty rows is forty chunks and fine. Four hundred thousand rows is not a document, it is a database, and it belongs behind a tool.

Strategy 2: load it and expose a tool

When questions span rows, the only correct answer is a query.

-- Load the workbook into a real table. It gets re-loaded on change; the
-- workbook stays the system of record and nobody edits this copy.
create table discount_tiers (
  region      text not null,
  tier        int  not null,
  discount    numeric(5,2) not null,
  min_order   numeric(12,2) not null,
  effective   date not null,
  loaded_at   timestamptz not null default now(),
  primary key (region, tier, effective)
);

export const discountTool = {
  name: 'lookup_discount_tiers',
  description:
    'Regional volume discount tiers: discount percentage and minimum order by ' +
    'region and tier. Use for any question about discounts, tiers or minimum ' +
    'order values, including comparisons between regions.',
  input: {
    region: 'string, optional. Omit to return all regions.',
    tier: 'integer, optional. Omit to return all tiers.',
  },
  async run({ region, tier }: { region?: string; tier?: number }, scope: Scope) {
    assertCanSee(scope, 'financial')   // discounts are financial. Chapter 13.

    const { rows } = await db.query(
      `select region, tier, discount, min_order, effective
         from discount_tiers
        where ($1::text is null or lower(region) = lower($1))
          and ($2::int is null or tier = $2)
          and effective <= current_date
        order by region, tier`,
      [region ?? null, tier ?? null],
    )
    // Returning the row count matters: it lets the answer say "across all six
    // regions" truthfully, and lets a complete-set claim be checked.
    return { rows, count: rows.length, complete: true }
  },
}

The complete: true flag is a small, high-value detail. It distinguishes a tool result that is the whole set from a retrieval result that is a sample, and it gives the model the basis for saying "all six regions" rather than implying it.

Strategy 3: both, which is usually the answer

Index serialised rows for lookups, expose a tool for aggregates, and index one generated summary document so that a question about the workbook's existence and shape retrieves something.

// A summary chunk, regenerated on every load. It answers "what pricing
// information is there?" — a question neither of the other strategies serves.
export function summariseTable(name: string, rows: Row[]): string {
  const regions = [...new Set(rows.map((r) => r.region))]
  const tiers = [...new Set(rows.map((r) => r.tier))].sort()

  return [
    `${name}`,
    ``,
    `Regional volume discount tiers covering ${regions.length} regions ` +
      `(${regions.join(', ')}) and ${tiers.length} tiers (${tiers.join(', ')}).`,
    `Discounts range from ${Math.min(...rows.map((r) => r.discount))}% to ` +
      `${Math.max(...rows.map((r) => r.discount))}%.`,
    `Minimum order values range from $${fmt(Math.min(...rows.map((r) => r.minOrder)))} ` +
      `to $${fmt(Math.max(...rows.map((r) => r.minOrder)))}.`,
    `For specific values, use the lookup_discount_tiers tool.`,
  ].join('\n')
}

The last line is doing real work. A retrieved summary that names the tool makes the model substantially more likely to call it rather than answering from the summary's ranges — a pattern worth reusing wherever an indexed document and a tool cover the same ground.

Tables inside documents

The other half of the problem: Atlas's product specifications are prose with tables in them, and the tables hold the specifications.

Naive extraction destroys them. A PDF table read as a text stream produces a run of values with the headers somewhere above, and a chunk boundary falling mid-table separates the two permanently.

// Two rules, both enforced in the chunker rather than hoped for.
export function chunkWithTables(blocks: Block[], targetTokens: number): Chunk[] {
  const chunks: Chunk[] = []
  let buffer: Block[] = []

  for (const block of blocks) {
    // Rule 1: a table is never split. An oversized table becomes its own
    // chunk even if that exceeds the target, because half a table is worse
    // than a large chunk.
    if (block.type === 'table') {
      flush()
      chunks.push(renderTable(block))
      continue
    }

    buffer.push(block)
    if (tokens(buffer) > targetTokens) flush()
  }
  flush()
  return chunks

  function flush() { /* ... as in chapter 9 ... */ }
}

// Rule 2: render tables as markdown, with the caption and heading path.
// Models read markdown tables reliably; they read pipe-less column runs badly.
function renderTable(block: TableBlock): Chunk {
  const header = `| ${block.headers.join(' | ')} |`
  const divider = `| ${block.headers.map(() => '---').join(' | ')} |`
  const body = block.rows.map((r) => `| ${r.join(' | ')} |`).join('\n')

  return {
    content: [block.caption, '', header, divider, body].filter(Boolean).join('\n'),
    headingPath: block.headingPath,
    kind: 'table',
  }
}

Keeping a kind on the chunk is worth it later: it lets you measure retrieval quality separately for tables and prose, and in practice those two numbers differ a lot. Chapter 48 does that measurement.

The failure mode: the average of six rows

This is the one to be able to recognise on sight, because it is invisible in the answer.

A rep asks: "what is our average volume discount across regions?" The workbook has been serialised into forty row-chunks. Retrieval returns twelve, because the window is twelve. The model computes the mean of twelve discounts and reports it as the average across regions.

The arithmetic is correct. The citation is real. The number is wrong, and no part of the answer indicates that twelve of forty rows were seen.

Two defences, and you want both.

// 1. Make the model's basis visible to it, so it can caveat honestly.
const sources = chunks
  .map((c, i) => `[${i + 1}] ${c.content}`)
  .join('\n\n')

const header = chunks.some((c) => c.kind === 'table_row')
  ? 'NOTE: The sources below are a selection of rows, not a complete set. ' +
    'Do not compute totals, averages or counts from them. If the question ' +
    'requires one, say that a complete lookup is needed.\n\n'
  : ''

# 2. An eval case, so the defence stays. This belongs in the set from chapter 10.
- id: agg-001
  question: "What is the average volume discount across regions?"
  asker: finance
  expect:
    # Either it calls the tool and gets it right, or it declines. Computing an
    # average from retrieved rows is a failure however plausible the number.
    one_of:
      - { must_contain: ["9.4%"], must_call_tool: ["lookup_discount_tiers"] }
      - { abstains: true }

The one_of shape is worth adopting generally. For questions where a decline is an acceptable outcome, encoding both acceptable answers stops the eval pushing the system toward confident invention.

Key points

Structure is not a format difference. A row means something only relative to its headers and often to other rows.
Serialise rows as self-describing key: value text. The retrieval improvement over pipe-delimited rows is large.
Row count decides the strategy: dozens of rows can be chunks; hundreds of thousands are a database behind a tool.
Most real workbooks need both, plus one generated summary chunk that names the tool.
Never split a table across chunks. Render as markdown, carry the caption and heading path, and tag the chunk kind.
Aggregation over retrieved rows produces correct arithmetic on an unstated sample. Tell the model its sources are a selection, and encode an eval case that accepts a decline.

Next: personal data, and what never enters the system — the decision that is much cheaper before the first ingest than after it.