Chapter 72 · Tools and actions

Confirmation flows: when a human must approve

A confirmation that is always approved is theatre. How to decide which actions need one, how to show enough that the answer is informed, and how to measure whether anyone is reading.

12 min read

Chapter 71 derived confirmation from reversibility. This chapter is about making the confirmation mean something.

The failure this chapter exists to prevent is specific and common: a confirmation that is approved 99.8% of the time has stopped being a control and become a click. It still appears in the architecture diagram, it still satisfies the reviewer who asked for it, and it prevents nothing.

When to require one

Three triggers, and only three. Everything else is noise that erodes the ones that matter.

export async function needsConfirmation(
  tool: Tool, args: unknown, scope: Scope, ctx: ResolvedContext,
): Promise<ConfirmationReason | null> {
  // 1. The action is observed or irreversible. Chapter 71.
  const level = await resolveReversibility(tool, args, scope)
  if (level === 'observed' || level === 'irreversible') {
    return { kind: 'irreversible', detail: describeEffect(tool, level) }
  }

  // 2. A value is unusual for this account. Chapter 69's plausibility check.
  const unusual = await detectUnusual(args, ctx)
  if (unusual) return { kind: 'unusual_value', detail: unusual }

  // 3. The action is at or near a limit the asker holds.
  if (nearLimit(args, scope.limits)) {
    return { kind: 'near_limit', detail: `at ${pct(ratio)} of your approval limit` }
  }

  return null
}

⚠️ Note what is not a trigger: low model confidence. Asking the model how sure it is and confirming when it is unsure was tried on Atlas and has the problem chapter 56 found — self-reported confidence tracks fluency rather than correctness, so it confirmed the hesitant correct actions and waved through the confident wrong ones.

Showing enough to make the answer informed

// ❌ A description of the arguments. Approving this is approving a
//    sentence, and the error is usually in a detail the sentence omits.
'Send a quote to Henderson for 200 units at 12% discount?'

// ✅ The artefact, with the fields that are usually wrong pulled out.
interface Confirmation {
  summary: string
  artefact: { kind: 'document' | 'record' | 'message'; rendered: string }
  keyFields: Array<{ label: string; value: string; unusual?: boolean }>
  alsoWill: string[]          // the side-effect chain, chapter 71
  reason: string              // why confirmation is required
}

Send quote QT-20994 to Henderson Industrial Supply?

  Recipient     j.oyelaran@henderson-industrial.com
  Account       Henderson Industrial Supply — Dallas (active)
  Product       SKU-4411, hydraulic seal 40mm
  Quantity      200
  Discount      12%        ⚠️ this account's usual rate is 8%
  Total         $18,400

  This will also: email the contact, and appear in the weekly quote report.
  Confirmation required because: sending a quote cannot be taken back.

  [ Send ]  [ Edit ]  [ Cancel ]

Three properties of that panel do the work.

The resolved entity is named, which is chapter 69's defence against the plausible wrong argument. Henderson Industrial Supply — Dallas is where a rep spots the wrong Henderson.

The unusual field is marked. Not a warning dialog — a marker on the one field that differs from this account's history. A rep's eye goes to it.

The reason is stated. "Cannot be taken back" tells the approver what they are being asked to carry.

The Edit option matters more than it looks: without it, a rep who spots the 12% has to cancel, retype the request, and hope. With it, the correction is one field and the confirmation re-renders.

Who may confirm

// The asker is not automatically the approver. A rep can draft a quote at
// any discount; confirming one above their limit is a different person.
export async function approverFor(
  tool: Tool, args: unknown, scope: Scope,
): Promise<ApproverRequirement> {
  const level = await resolveReversibility(tool, args, scope)

  if (level === 'irreversible') {
    return { kind: 'second_person', capability: tool.requires!,
             excluding: [scope.userId] }   // ⚠️ never the initiator
  }
  if (exceedsLimit(args, scope.limits)) {
    return { kind: 'escalate', toRole: escalationRole(scope.role),
             reason: `exceeds your limit of ${scope.limits.discountPercent}%` }
  }
  return { kind: 'self' }
}

The excluding: [scope.userId] is the whole point of a second approver and it is easy to omit. An approval flow where the initiator can approve their own action is a log entry, not a control.

The timeout, and what it must not do

// A pending confirmation is state. It expires, and expiry means NOT done.
export const CONFIRMATION_TTL_MINUTES = 30

export async function resolveConfirmation(token: string, decision: 'approve' | 'reject') {
  const pending = await getPending(token)
  if (!pending) return { ok: false, error: 'unknown_or_expired' }

  if (Date.now() - +pending.createdAt > CONFIRMATION_TTL_MINUTES * 60_000) {
    await markExpired(pending.id)
    // ⚠️ Expired means the action did NOT happen, and the preview may now
    // be stale — the order could have changed in thirty minutes. Re-run
    // the whole flow rather than executing an old decision.
    return { ok: false, error: 'expired',
             message: 'This confirmation expired. Please ask again.' }
  }
  // ...
}

// And the check that catches the subtle version: approved, but the world
// moved between preview and approval.
const current = await tool.preview(pending.arguments, pending.scope)
if (previewDigest(current) !== pending.previewDigest) {
  return { ok: false, error: 'changed',
    message: 'The details changed since you were shown this. Please review again.',
    diff: diffPreviews(pending.preview, current) }
}

An approval is consent to a specific thing. If the thing changed, the consent does not transfer — and on Atlas this fired 0.4% of the time, mostly orders whose status advanced while the rep was in a meeting.

Confirmation and the loop

Chapter 66 exited the loop on a confirmation request. The reason is worth stating directly.

// ❌ Returned as a tool result, the model reasons around it. Observed
//    continuations: calling the tool again, calling a different tool with
//    the same effect, or telling the user it has been sent.
results.push({ tool_use_id: call.id, content: 'Confirmation required.' })

// ✅ The loop ends. The run is suspended, the state is persisted, and
//    approval resumes it as a new run with the decision as a fact.
return { kind: 'needs_confirmation', pending: call, preview, trace }

// Resumption, after approval. The model is told the outcome, not asked
// to decide again.
export async function resumeAfterConfirmation(runId: string, decision: Decision) {
  const run = await loadRun(runId)
  const result = decision === 'approve'
    ? await executeToolRequest(run.pending, run.scope, { ...run.context, confirmed: true })
    : { ok: false, error: 'declined_by_user' }

  return runToolLoop(run.messages.concat(toolResultMessage(run.pending, result)),
                     run.scope, run.limits)
}

Measuring whether it is a control

The number that tells you whether any of this is working.

select tool,
       count(*) as requested,
       round(100.0 * count(*) filter (where decision = 'approve') / count(*)) as approved_pct,
       round(100.0 * count(*) filter (where decision = 'reject') / count(*)) as rejected_pct,
       round(100.0 * count(*) filter (where decision = 'edit') / count(*)) as edited_pct,
       percentile_cont(0.5) within group (order by decided_at - created_at) as median_time
  from confirmations
 where created_at > now() - interval '30 days'
 group by 1 order by requested desc;

Atlas, three months in:

Tool	Requested	Approved	Rejected	Edited	Median time
`send_quote`	412	89%	4%	7%	24 s
`apply_credit`	38	76%	13%	11%	71 s
`update_order`	203	99.5%	0.5%	0%	3 s

The third row is the failure. 99.5% approval with a three-second median means nobody is reading it, and update_order was being confirmed on every call because its reversibility was declared as a constant rather than state-dependent.

Fixing that reduced update_order confirmations from 203 to 19 — the orders actually in picking — and the approval rate on those dropped to 84%.

A confirmation rate above about 97% means the trigger is too broad. The fix is to confirm less, not to make the dialog louder.

The edited_pct column is the one to watch most closely. 7% edits on send_quote is 29 quotes that would have gone out wrong, and it is the clearest evidence the control is doing something.

Batch confirmation, carefully

// ⚠️ A list of twenty actions approved with one click is one confirmation
//    for twenty effects. Allowed only when the items are homogeneous and
//    each is individually shown.
export function canBatch(requests: ToolRequest[]): boolean {
  if (requests.length > 10) return false
  if (new Set(requests.map((r) => r.tool)).size > 1) return false
  return !requests.some((r) => TOOLS[r.tool].reversibility === 'irreversible')
}

Irreversible actions are never batched. The point of the control is attention on each one.

The failure mode: the confirmation that trained the user

open_ticket is given a confirmation during the rollout, out of caution. It is reversible, internal, and low-stakes.

Reps see it fifteen times a day. Within a fortnight the approval is reflexive — the panel appears, the hand clicks, and the content is not read.

Then send_quote ships, with a confirmation that looks the same.

The habit transfers. The send_quote panel is read with the same attention as the ticket panel, which is none, and the first wrong-recipient quote goes out in week three.

// 1. Do not confirm low-stakes actions. The cost is not the click, it is
//    the habit. open_ticket's confirmation was removed.

// 2. Make the high-stakes panel visually distinct and slower.
interface Confirmation {
  severity: 'standard' | 'high'
  // ⚠️ A deliberate delay before the approve button enables. Two seconds
  // is enough to break a reflex and short enough not to annoy.
  minimumReviewMs?: number
}

// 3. Require an explicit gesture proportional to the stakes.
//    'click' for standard; 'type_to_confirm' for irreversible.
confirmationGesture: level === 'irreversible' ? 'type_to_confirm' : 'click'

Apply a credit of $4,200 to Henderson Industrial Supply?

  ⚠️ This cannot be undone. A reversal is a separate entry on the account.

  Type the account name to confirm:  [________________]

Atlas's measurement after separating the tiers: apply_credit rejection rose from 2% to 13%, and median decision time from 8 seconds to 71. The same people, the same actions — the difference is that the panel asked for attention and got it.

Every low-stakes confirmation you add makes every high-stakes confirmation weaker. Spend them like a budget.

Key points

A confirmation approved 99.8% of the time is theatre that still appears in the architecture diagram.
Three triggers only: irreversible or observed, an unusual value for this account, near the asker's limit. Model confidence is not one.
Show the artefact with the usually-wrong fields pulled out, the unusual one marked, the side effects listed, and the reason stated.
Offer Edit. Without it, spotting an error means starting over.
The initiator is never the second approver.
A confirmation expires, and an expired one means the action did not happen. Re-check the preview on approval: consent is to a specific thing.
The confirmation must exit the loop. Returned as a tool result, the model reasons around it.
Approval above ~97% means the trigger is too broad. Confirm less rather than shouting louder.
Track the edit rate — it is the clearest evidence the control is working.
Never batch irreversible actions.
Low-stakes confirmations train the reflex that defeats high-stakes ones. Separate the tiers visually, with a delay and a stronger gesture.

Next: idempotency for agent actions — the action taken twice.