Chapter 147 · Interface and adoption

Setting expectations so trust survives the first wrong answer

There will be a wrong answer. Whether it ends the relationship depends on what was promised, the asymmetry in how trust moves, and the four sentences that do most of the work.

7 min read

Chapter 110 measured Atlas at 1.4% invention and 4.8% grounded-but-wrong. Over a year at 1,300 questions a week, that is roughly four thousand wrong answers. The system is good, and people will encounter wrongness.

What decides whether that ends the relationship is almost entirely what they were told beforehand, and the asymmetry is brutal.

How trust moves

// Measured by tracking individual users' question volume before and
// after their first encounter with a wrong answer.
const TRUST = {
  correctAnswersToBuild: '~15 before someone relies on it without checking',
  wrongAnswersToLose: 1,
  recoveryAfterOne: '~40 correct answers, if they keep using it at all',
  whoKeepsUsingIt: {
    toldItCanBeWrong: 0.81,
    notTold: 0.34,
  },
}

Fifteen to build, one to lose. And the only variable in the bottom row is whether anyone had said, in advance, that this would happen.

The mechanism is not about forgiveness. A user who was told the system can be wrong and shown how to check an answer has a procedure when it happens: they check, they find the error, they flag it, and the event confirms the model they were given. A user who was told it is accurate has no procedure, and the event destroys the model entirely.

Being told something can fail is not a warning. It is the instruction for what to do when it does.

Four sentences

// The whole expectation-setting programme, in four sentences placed
// where they cannot be missed.
const SENTENCES = {
  scope:  'I can see the policy library, supplier contracts, the orders ' +
          'system and inventory. I cannot see HR, financial reporting, ' +
          'or anything in personal drives.',
  fallibility: 'I can be wrong. Every claim links to the sentence it ' +
          'came from, so you can check in a few seconds.',
  provenance: 'I answer from your company’s documents, not from general ' +
          'knowledge. If I have no document, I will say so.',
  authority: 'For anything you are going to commit to a customer, check ' +
          'the source. I am a faster way to find it, not the authority.',
}

// Placed, in order of how many people see them:
//   scope       -> the empty state, every session (chapter 144)
//   fallibility -> the first-session hint (chapter 146)
//   provenance  -> the uncited-answer banner (chapter 143)
//   authority   -> the footer under every answer, permanently

The fourth is the one Atlas debated longest, because it reads as undermining the product. It is the opposite: it names what the system actually is — a faster path to the source — and that framing survives a wrong answer intact. A system positioned as the authority has nowhere to go when it is wrong.

Measured: adding the authority line reduced self-reported reliance on uncited answers from 44% to 19%, and had no effect on usage volume.

Promises that cost you

// Three claims that are true, sound good, and create a liability.
const DANGEROUS = {
  'instant answers':    'sets a latency expectation chapter 133 cannot meet at p95',
  'knows everything about the company': 'guarantees a disappointment on the first ' +
    'question about something that is not indexed',
  'always cites its sources': 'true 97.4% of the time, and the 2.6% now reads ' +
    'as a broken promise rather than a marked exception',
}

The third is the shape to watch for generally: a promise with an exception rate is worse than a promise with a stated condition. "Always cites" invites the exception to be read as a bug. "Answers from your documents, and tells you when it has not" makes the same 2.6% a feature of the design.

The first wrong answer, handled

// It will arrive. What happens in the next ten minutes decides the
// outcome more than anything in the preceding year.
const RESPONSE = [
  '1. The claim-level flag is one click away. Chapter 145.',
  '2. The run id is visible, so the report is actionable. Chapter 128.',
  '3. Someone replies within a day, with what was wrong and why.',
  '4. The closing notice, when it is fixed. Chapter 145’s 74%.',
]

Atlas's data on users who reported a wrong answer:

	Still using it a month later
Reported, no reply	38%
Reported, got a reply	72%
Reported, got a reply and a closing notice	91%

Ninety-one percent, among the people who had just been given a wrong answer — higher than the overall base rate. A handled failure builds more trust than an absence of failures, which is an old finding in service design and holds exactly here.

Expectations for the people it replaces

// A different audience with a different fear, and the honest version
// is the only one that works. Chapter 146 put the operations team in
// the third rollout wave for this reason.
const OPS_MESSAGE = {
  wrong: '"It will not replace you." Nobody believes it, and it is not ' +
         'the question they are asking.',
  right: '"It answers the 60% of questions that are a lookup. The ones " +' +
         'that reach you will be the harder ones. Here is the escalation ' +
         'rate, and we will watch it with you."',
}

Giving the operations team the escalation metric as a shared number rather than a management one changed the relationship. They became the people most likely to report a bad answer, which is exactly who you want reporting them — and chapter 145's finding that 41% of feedback was about wrong documents was largely their work.

The thing never to promise

// One claim that is always wrong to make, however tempting under
// pressure from someone senior.
//   ❌ "It will never say anything wrong to a customer."
//
// It cannot be guaranteed, the guarantee is the thing that gets
// quoted back, and the honest version is stronger: every claim is
// checkable in four seconds, we measure how often it is wrong, and
// here is this quarter's number.

A measured number offered unprompted does more for trust with a senior stakeholder than any assurance, and it is the same principle as chapter 126's advice for a security review: name the measurement, name the control, offer the artifact.

Key points

At Atlas's measured error rates, a year produces roughly four thousand wrong answers. People will encounter wrongness.
Fifteen correct answers build reliance; one wrong answer removes it. The only variable is whether they were told in advance.
81% of users who were told it can be wrong kept using it after encountering one, against 34% who were not.
Being told something can fail is not a warning. It is the instruction for what to do when it does.
Four sentences: scope, fallibility, provenance, authority. Place them in the empty state, the first-session hint, the uncited banner and the permanent footer.
The authority line — "a faster way to find it, not the authority" — cut reliance on uncited answers from 44% to 19% with no effect on volume.
A promise with an exception rate is worse than a promise with a stated condition. "Always cites" makes the 2.6% a broken promise.
Handle the first wrong answer well: 38% retention with no reply, 72% with a reply, 91% with a closing notice — above the base rate.
For the team whose work it absorbs, share the escalation metric rather than reassurance. They become the best reporters of bad answers.
Never promise it will not be wrong. Offer the measurement instead.

Next: measuring adoption against usage — the numbers that say whether any of this worked.