Chapter 66 · Tools and actions

Tool calling, and how the model decides

What actually happens when a model calls a tool, the five failure points in that loop, and why the decision is made almost entirely from text you wrote.

11 min read

Before designing a tool surface it is worth being precise about the mechanism, because several design rules in this section follow directly from it and look arbitrary otherwise.

The loop

Four things about that loop decide most of this section.

The model never executes anything. It emits a structured request. Every guard in chapter 65's five gates sits between the request and any effect.

Each turn is a full model call. A three-tool chain is four calls — three to request, one to conclude — with the whole conversation resent each time. That is the cost structure behind chapter 47's routing argument.

The result is text the model reads. Whatever your tool returns becomes context. Chapter 77 is entirely about this.

The decision is made from the tool definitions. Name, description, parameter names and descriptions — nothing else. The model cannot see your implementation.

What the model actually sees

// This is the complete input to the decision. If a tool is chosen wrongly,
// the cause is in these ~60 tokens.
{
  name: 'check_inventory',
  description:
    'Current on-hand quantity for a SKU, optionally at one warehouse. Use for ' +
    'any question about stock levels, availability or quantities on hand.',
  input_schema: {
    type: 'object',
    properties: {
      sku: { type: 'string', description: 'The SKU code, e.g. SKU-4411.' },
      warehouse: {
        type: 'string',
        enum: ['dallas', 'reno', 'columbus'],
        description: 'Optional. Omit to return all warehouses.',
      },
    },
    required: ['sku'],
  },
}

The tool description is not documentation. It is the entire basis on which the model chooses, so it is a prompt — and it gets the same treatment as any other prompt: versioned, reviewed, tested.

Chapter 68 is about writing them well. This chapter is about what the loop does with them.

Implementing the loop

// src/agent/loop.ts — bounded, auditable, and the bounds are not optional.
export async function runToolLoop(
  question: string, scope: Scope, limits: Limits,
): Promise<LoopResult> {
  const messages: Message[] = [{ role: 'user', content: question }]
  const trace: TraceEntry[] = []
  const context: RunContext = { called: [], startedAt: Date.now() }

  for (let step = 0; step < limits.maxSteps; step++) {
    // ⚠️ Time and spend limits checked every step, not only at the end.
    // A loop that exits after eight steps has already spent eight calls.
    if (Date.now() - context.startedAt > limits.maxMs) {
      return { kind: 'timeout', trace, partial: messages }
    }

    const response = await callModel({
      messages,
      tools: selectTools(question, scope, { maxTokens: BUDGET.tools }),
      model: modelFor(question, { multiHop: step > 0 }),
    })

    if (response.stopReason !== 'tool_use') {
      return { kind: 'complete', text: response.text, trace }
    }

    messages.push({ role: 'assistant', content: response.content })

    // Several tool_use blocks can arrive at once. Chapter 88 on whether
    // running them in parallel is safe; here they run in order.
    const results = []
    for (const call of response.toolCalls) {
      const result = await executeToolRequest(call, scope, context)
      context.called.push(call.name)
      trace.push({ step, tool: call.name, arguments: call.arguments, result })

      // ⚠️ A confirmation request exits the loop entirely. It does not
      // become a tool result the model reasons around.
      if (result.kind === 'needs_confirmation') {
        return { kind: 'needs_confirmation', pending: call, preview: result.preview, trace }
      }
      results.push({ tool_use_id: call.id, content: renderToolResult(result) })
    }

    messages.push({ role: 'user', content: results })
  }

  return { kind: 'step_limit', trace }
}

Two details in that function are load-bearing.

A confirmation request returns rather than continuing. If it were returned to the model as a tool result, the model would reason about it — and the obvious continuation is to call the tool again or find another route to the same effect. Chapter 72 depends on this exit.

Limits are checked every step. A maxSteps of 8 checked only at the top is still 8 model calls; the time and spend checks are what stop a slow chain, and they have to be inside the loop.

The five failure points

#	Point	Symptom	Chapter
1	Chooses no tool when it should	Answers from retrieval or from nothing	67
2	Chooses the wrong tool	Plausible answer from the wrong system	68
3	Constructs wrong arguments	Right action, wrong object	69
4	Misreads the result	Correct call, wrong conclusion	77
5	Does not stop	Loops until the step limit	76, 80

Atlas's distribution over 400 tool-using questions:

tool selection correct     0.99   (after per-request filtering)
arguments correct          0.91   ⚠️
result interpreted right   0.96
terminated correctly       0.98
end to end                 0.86

Three things to read from that.

0.91 on arguments dominates. It is the worst number and it compounds into everything else — which is why chapter 69 is the longest chapter in this section.

End to end is the product, not the minimum. 0.99 × 0.91 × 0.96 × 0.98 ≈ 0.85, and the measured 0.86 matches. Four stages at 96% average is 86% overall, and that multiplication is the thing to internalise about tool loops.

Two percent not terminating is not a quality statistic. Each one is a run that burned the step limit, and at eight steps that is eight model calls for no answer.

Tool choice is a prompt problem

The strongest lever on failure points 1 and 2 is not the model. It is the surface.

// Measured on Atlas, same model and cases throughout.
// 14 tools, no guidance              correct tool 0.89
// 14 tools, overlap removed          correct tool 0.94   (chapter 67)
// 14 tools, descriptions rewritten   correct tool 0.97   (chapter 68)
// 4 relevant tools, filtered         correct tool 0.99   (chapter 67)

Ten points from four changes, none of which touched the model. The largest single gain was removing two tools whose descriptions overlapped — search_tickets and get_ticket_history both plausibly answered "has this come up before?", and the model split between them roughly at random.

Forcing and forbidding

Two controls worth knowing, both used sparingly.

// Force a specific tool when the route is already classified. It removes
// failure point 2 entirely for that request.
const response = await callModel({
  messages, tools,
  toolChoice: { type: 'tool', name: 'check_inventory' },
})

// Or forbid tools entirely, for a request the router classified as a
// pure document question. Cheaper and removes a class of failure.
const response = await callModel({ messages, tools: [], toolChoice: 'none' })

⚠️ Forcing is tempting and it moves the failure rather than removing it: a forced tool on a misrouted question produces a confident call against the wrong system. Force only where the router is measurably reliable for that class — on Atlas that is three question shapes with pattern-level precision above 0.99.

Observing the loop

// Every step, logged. Without this a bad outcome is unattributable among
// five failure points.
logger.info('tool_loop.step', {
  runId, step, tool: call.name,
  arguments: redactArguments(call.arguments),   // ⚠️ may contain account data
  resultKind: result.ok ? 'ok' : result.error,
  latencyMs, tokensIn, tokensOut,
})

-- The query that finds failure point 5 before anyone complains.
select tool, count(*) as calls,
       round(avg(step), 1) as avg_step,
       count(*) filter (where step >= 6) as late_calls
  from tool_calls
 where called_at > now() - interval '7 days'
 group by 1 order by late_calls desc;

A tool appearing mostly at step 6 or later is a tool the model reaches for when it is stuck, which usually means an earlier tool returned something unhelpful — failure point 4 wearing failure point 5's clothes.

The failure mode: the tool that was never called

The quietest failure in this chapter. lookup_discount_tiers is defined, correct, and tested. Over a month it is called eleven times, and discount questions are 7% of traffic.

The model is answering them from retrieval instead, using the pricing workbook's indexed rows. Those answers are plausible, cited, and from last night's snapshot — and on the questions needing a comparison across regions they are the average of six rows.

Nothing failed. A path existed that looked sufficient, so the tool was never chosen.

// 1. Detect it: a tool with a known question share and no calls.
export async function toolUtilisation() {
  const expected = { lookup_discount_tiers: 0.07, check_inventory: 0.11 }
  for (const [tool, share] of Object.entries(expected)) {
    const actual = await callShare(tool, { days: 30 })
    if (actual < share * 0.3) {
      logger.warn('tool.underused', { tool, expected: share, actual,
        note: 'Questions may be answered from retrieval instead.' })
    }
  }
}

// 2. Fix the cause, which is in the indexed source rather than in the tool.
//    Chapter 15 already recommended this and it matters most here: the
//    summary chunk names the tool.
`For specific values or comparisons across regions, use the ` +
`lookup_discount_tiers tool.`

// 3. ⚠️ The structural fix: do not index content a tool answers better.
//    The indexed row-chunks were suppressed, leaving only the summary.
//    Discount tool calls went from 11/month to 340, and the "average of
//    six rows" failure disappeared because the six rows are no longer
//    retrievable.

A tool competes with retrieval for the same question. If the indexed version is good enough to look right, the tool will not be called — and the fix is usually to remove the indexed version, not to improve the tool.

Key points

The model emits a request; it never executes. Every guard sits between request and effect.
Each turn is a full model call with the whole conversation resent. A three-tool chain is four calls.
The decision is made entirely from the tool definitions, so a description is a prompt and gets a prompt's treatment.
A confirmation request must exit the loop, not return as a tool result the model can reason around.
Check time and spend limits inside the loop, not only the step count.
Five failure points, and they multiply. Four stages at 96% average is 86% end to end.
Argument construction is the worst stage at 0.91 and dominates the product.
Ten points of tool-selection accuracy came from the surface — removing overlap, rewriting descriptions, filtering per request — with no model change.
Force a tool only where the router is measurably reliable; otherwise forcing moves the failure.
A tool competes with retrieval. If the indexed version looks sufficient, suppress it rather than improving the tool.

Next: designing a tool surface a model can actually use.