Chapter 81 · Agent loops and control

Step limits, time limits and spend limits

Three bounds that fail differently and are all needed. Where to set each one, the global limits that bound a bad deploy, and the budget object that makes them checkable in one line.

10 min read

Chapter 80 treated limits as a backstop and argued for detecting non-termination earlier. This chapter is the backstop itself, because it still has to be right.

Three bounds, and the mistake is thinking one of them subsumes the others.

Why three

Bound	Catches	Misses
Steps	A loop that will not converge	A single 40-second tool call
Time	A slow run, whatever the cause	A fast run that calls an expensive model six times
Spend	An expensive run	A cheap run that takes four minutes

// A concrete case for each, all of which Atlas has seen.
// 2 steps, 71 seconds:   a database tool without a statement timeout
// 3 steps, $0.84:        the planning model with a 40k-token context
// 9 steps, 4 seconds:    an oscillation on a fast cached tool

Each bound catches a case the others let through, so all three are needed and each is cheap.

The budget object

// src/agent/budget.ts — one object, checked in one line per step.
export interface Limits {
  maxSteps: number
  maxMs: number
  maxSpendUsd: number
  maxToolCalls: number        // ⚠️ distinct from steps: one step can emit several
}

export class Budget {
  private spendUsd = 0
  private toolCalls = 0
  private readonly startedAt = Date.now()
  private step = 0

  constructor(private readonly limits: Limits) {}

  /** Checked BEFORE the model call, per chapter 79. */
  check(): HaltReason | null {
    if (this.step >= this.limits.maxSteps) return 'step_limit'
    if (Date.now() - this.startedAt >= this.limits.maxMs) return 'time_limit'
    if (this.spendUsd >= this.limits.maxSpendUsd) return 'spend_limit'
    if (this.toolCalls >= this.limits.maxToolCalls) return 'tool_call_limit'
    return null
  }

  record(usage: Usage, model: string) {
    this.spendUsd += costOf(usage, model)
    this.step++
  }

  recordToolCalls(n: number) { this.toolCalls += n }

  /** What remains, for sizing a tool's timeout. Chapter 75. */
  remainingMs(): number {
    return Math.max(0, this.limits.maxMs - (Date.now() - this.startedAt))
  }
}

maxToolCalls separate from maxSteps matters because a model can emit several tool_use blocks in one turn. Eight steps with four parallel calls each is 32 tool calls, which is a different blast radius from eight.

Setting the numbers

Derive them from the targets, not from intuition.

// Atlas, derived. Every number has a reason beside it.
export const LIMITS: Limits = {
  // p95 latency target is 10s. Measured: a step is ~1.4s including tools,
  // and generation of the final answer is ~1.5s. (10 - 1.5) / 1.4 ≈ 6.
  maxSteps: 6,

  // The target itself, as a hard stop. A run at 10s has already failed
  // the SLO; continuing only makes it worse.
  maxMs: 10_000,

  // ⚠️ 20x the median run cost. Not a cost control — a blast-radius bound.
  // A run costing 20x the median is pathological regardless of its budget.
  maxSpendUsd: 0.40,

  maxToolCalls: 12,
}

// Per-route overrides, from chapter 80's finding that a global raise is
// a cost increase for the 97% it does not help.
export function limitsFor(route: Route): Limits {
  if (route.kind !== 'planning') return { ...LIMITS, maxSteps: 2, maxMs: 6_000 }
  if (route.complexity === 'high') return { ...LIMITS, maxSteps: 10, maxMs: 20_000 }
  return LIMITS
}

⚠️ The high-complexity route breaks the p95 target deliberately, and that is a decision rather than an oversight. A past-due investigation taking 18 seconds is acceptable to a rep who asked a hard question; the same wait on a policy lookup is not. Chapter 142 covers showing progress while it runs.

What a halt costs

The reason spend limits are a blast-radius bound rather than a cost control.

// Measured on Atlas, 30 days.
// Total loop spend:              $341
// Spend on runs that answered:   $312   (91%)
// Spend on halted runs:          $29    (9%)
// ⚠️ Spend on step_limit runs:   $0.59  (0.2%) — seven runs

The seven step-limit runs cost 59 cents. Tightening the limit would save pennies; the value of the bound is that it prevents a pathological run rather than that it saves money.

A spend limit is not for the bill. It is so that a bug cannot turn one request into a hundred dollars while nobody is watching.

The limits that actually matter: global

Per-run limits bound one request. A bad deploy affects every request, and a per-run limit does nothing about it.

// src/agent/circuit.ts — the bounds that stop an incident.
export const GLOBAL_LIMITS = {
  // Loop runs per hour, across all users. ~3x the observed peak.
  runsPerHour: 400,
  // Spend per hour. ~5x the observed peak.
  spendPerHourUsd: 12,
  // ⚠️ The most useful one: average steps per run. A deploy that breaks
  // termination shows up here within minutes, long before the spend does.
  avgStepsPerRun: 4.0,
}

export async function checkGlobalHealth() {
  const window = await loopStats({ minutes: 15 })

  if (window.avgSteps > GLOBAL_LIMITS.avgStepsPerRun) {
    // Degrade rather than stop: fall back to the cheap path, which still
    // answers 71% of questions.
    await setLoopEnabled(false)
    alert(`loop disabled: avg steps ${window.avgSteps.toFixed(1)} ` +
          `exceeds ${GLOBAL_LIMITS.avgStepsPerRun}`)
  }
  if (window.spendPerHour > GLOBAL_LIMITS.spendPerHourUsd) {
    await setLoopEnabled(false)
    alert(`loop disabled: spend rate $${window.spendPerHour.toFixed(2)}/hour`)
  }
}

// Degradation, not failure. A disabled loop answers from retrieval and
// says what it could not do.
if (!(await isLoopEnabled())) {
  const answer = await cheapPath(question, scope)
  answer.caveats.push('Detailed investigation is temporarily unavailable.')
  return answer
}

Atlas's global limits have fired twice, both on deploys: once when termination nudging was accidentally disabled and average steps went from 2.4 to 5.8 in eleven minutes, and once when a tool's timeout was removed.

The average-steps trigger caught both, and it caught them before the spend trigger would have — which is why it is the one to watch.

Per-user limits

// Bound one user's share, so one person's enthusiasm cannot consume the
// global budget. Generous: this is an abuse bound, not a usage policy.
export const PER_USER_LIMITS = {
  runsPerHour: 30,
  spendPerDayUsd: 2.00,
}

⚠️ A per-user limit that users actually hit is a product problem, not a safety control. On Atlas no user has hit either, and if one did the correct first response would be to ask why they need thirty investigations an hour.

Reporting a halt honestly

// Each reason gets its own message. A generic "that took too long" tells
// the user nothing about whether to retry.
const HALT_MESSAGES: Record<HaltReason, string> = {
  step_limit:
    'I could not work this out in the steps available. Here is what I found.',
  time_limit:
    'This took longer than expected and I stopped. Here is what I found so far.',
  spend_limit:
    'I stopped this investigation early. Here is what I found.',
  tool_call_limit:
    'I checked several systems and stopped. Here is what I found.',
}

// ⚠️ And none of them says "spend limit" to the user. The reason is
// internal; the user needs the partial answer and whether to retry.
return { ...(await answerFromPartial(run, reason)),
         internalReason: reason,          // logged, not shown
         retryable: reason === 'time_limit' }

Marking time_limit as retryable and the others not is a small honesty: a slow dependency may be fine in a minute, and a run that hit the step limit will hit it again.

The failure mode: the limit checked after the work

The subtle one, and it was in Atlas's first implementation.

// ❌ The check happens after the call. A run at the step limit has
//    already paid for the step that exceeded it — and with the
//    quadratic cost growth, that last step is the expensive one.
while (true) {
  const response = await callModel(...)
  budget.record(response.usage)
  if (budget.check()) break            // ⚠️ too late
  // ...
}

// ✅ Check first. A run halts having spent exactly its budget, not its
//    budget plus the most expensive step.
while (true) {
  if (budget.check()) break
  const response = await callModel(...)
  budget.record(response.usage)
  // ...
}

The difference on Atlas was 18% of total loop spend, because the final step of a deep run is the largest.

There is a second, worse version of the same mistake, in the time bound:

// ❌ The time limit is checked, but the call it permits has no timeout
//    of its own. A run at 9.8s against a 10s limit starts a model call
//    that takes 4 seconds, and the run finishes at 13.8s.
if (Date.now() - startedAt >= maxMs) break
const response = await callModel(...)            // unbounded

// ✅ The call is bounded by what remains, per chapter 75.
const response = await callModel({ ..., signal: AbortSignal.timeout(budget.remainingMs()) })

A limit checked before an unbounded operation is a suggestion. Bound the operation by what remains, or the limit is only ever approximately honoured.

Key points

Three bounds, each catching a case the others miss. All three are cheap.
Count tool calls separately from steps: one step can emit several.
Derive the numbers from the latency target, not from intuition, and record the reason beside each.
Override per route rather than globally. A high-complexity route may break the p95 target deliberately.
A spend limit is a blast-radius bound, not a cost control — halted runs were 0.2% of Atlas's loop spend.
Global limits are what bound a bad deploy. Average steps per run is the most sensitive trigger and fires before spend does.
Degrade to the cheap path rather than failing. It still answers most questions.
A per-user limit that users hit is a product problem, not a safety control.
Give each halt reason its own message, never expose the internal reason, and mark only time_limit as retryable.
Check the bound before the work, and bound the operation by what remains. Otherwise the limit is approximate and the overrun is the expensive step.

Next: explicit plans against implicit ones.