Chapter 146 · Interface and adoption

Onboarding people who have never used one

The documentation 12% of people read, what the first five minutes have to accomplish, and the rollout order that took Atlas from 34% to 71% repeat use without any change to the model.

7 min read

Chapter 145 found nine people who gave up on something the system could do. This chapter is about reaching them in the first five minutes, which is the only window that exists.

The constraint that shapes everything: a user forms a model of what the system can do from their first two or three questions, and that model is durable. A rep whose first question failed believes it cannot do that category of thing, and will not retry in six months when it can.

What documentation achieves

// Atlas wrote a good one: a page on the intranet, what it can do, how
// to phrase a question, examples.
const DOCS = {
  readBeforeFirstUse: 0.12,
  readEver: 0.19,
  repeatUseAmongReaders: 0.68,      // it works, for the 12%
  repeatUseAmongNonReaders: 0.31,
}

The document works and almost nobody reads it. That is not a failure of the document, it is the normal response to optional reading about an internal tool, and the lesson is to move its content into places people cannot avoid.

// Where the document's content actually went.
const RELOCATED = {
  'what it can do':      'the empty-state scope line — chapter 144',
  'how to phrase':       'the structured starters — chapter 141',
  'what it cannot do':   'the failure screens — chapter 144',
  'how to check it':     'the citation panel — chapter 143',
  'how to report':       'the claim-level control — chapter 145',
}

Every row is a chapter that already exists. The onboarding was mostly a relabelling of work done for other reasons, which is the practical shape of this chapter: the interface is the onboarding, and the document is for the 12%.

The first five minutes

// Three things, in order. Anything else competes with them.
const FIRST_FIVE = [
  '1. One question that works, chosen by them, not demonstrated to them.',
  '2. One citation opened, so they see it is checkable.',
  '3. One honest refusal, so they learn the boundary while stakes are low.',
]

The third is counter-intuitive and was the highest-leverage change Atlas made. A user who meets their first refusal in week three, on a question that mattered, concludes the system is unreliable. A user who meets one in minute four, on a question they were idly curious about, learns the scope.

// Which is a deliberate design: the starter set includes one entry
// that is answered by a refusal, and the refusal is a good one.
{ label: 'Check a salary band', template: 'What is the salary band for ___?' }
// -> "I don't have access to HR records. Compensation questions go to
//     People Ops. Here is what I CAN see: [scope]"

Including a deliberately-refused starter feels wrong and measured well: repeat use among users who hit it in their first session was 79%, against 64% for those who did not.

Rollout order

// Not everyone at once. The order Atlas used, and the reasoning.
const WAVES = [
  { who: 'the 6 people who asked for it', when: 'week 0', why: 'motivated; tolerant of rough edges' },
  { who: 'one team, fully',               when: 'week 3', why: 'a real distribution, and they talk to each other' },
  { who: 'the operations team',           when: 'week 6', why: '⚠️ the people it partly replaces. Last of the pilots, deliberately' },
  { who: 'everyone',                      when: 'week 10', why: 'after the starters were rewritten from wave 2’s traffic' },
]

// The operations team placement is the judgement call. They are the
// experts whose work it absorbs, and they are both the harshest
// reviewers and the people whose endorsement decides whether anyone
// else trusts it. Going to them before it was good enough would have
// been fatal; going after it had been tested elsewhere made them
// advocates.

Between wave 2 and wave 4, the starters were rewritten entirely from the actual question distribution of a real team — which is the reason the full rollout landed at 71% repeat use against the pilot's 34%.

Teaching by answering

// The most effective onboarding is attached to answers, not to a tour.
// A tour is dismissed; an answer is read.
<Answer>
  ...
  <Hint dismissible once>
    Sources are listed under each claim. Click one to see the exact
    sentence it came from.
  </Hint>
</Answer>

// Five hints, each shown once, each attached to the moment it is
// relevant. Dismissal is permanent and they never reappear.
const HINTS = [
  { after: 'first answer with citations',   text: 'how to check a source' },
  { after: 'first refusal',                 text: 'what it can and cannot see' },
  { after: 'first multi-turn session',      text: 'it remembers the current topic' },
  { after: 'first slow run',                text: 'what the step list means' },
  { after: 'first wrong-looking answer',    text: 'how to flag a specific claim' },
]

Atlas's hints are seen by 94% of users, against the document's 19%, because they arrive at the moment the thing they describe is on screen. A tour at first login was tried and dismissed within four seconds by 71% of people.

What not to do

const AVOID = [
  'A demo of the most impressive thing it can do. It sets an expectation ' +
    'the median question will not meet, and chapter 147 is about the cost.',
  'Training sessions. 40 minutes for a tool whose whole point is that it ' +
    'needs no training. Attendance was 22% and repeat use among attendees ' +
    'was no different.',
  'Calling it AI in the launch message. It invites comparison to a ' +
    'general assistant. Atlas is described as "search and answers over our ' +
    'own documents", which is accurate and sets the right scope.',
  'Rolling out to everyone on day one, before the starters reflect real traffic.',
]

The third is a naming decision with a measurable effect. The launch message describing it as an AI assistant produced a first-week out-of-scope rate of 24%; the rewritten one describing it as search and answers over company documents produced 11%.

People do not need to be taught to ask a question. They need to be told what this particular thing knows, and shown once that the answer can be checked.

Key points

A user's model of what the system can do forms in their first two or three questions and is durable. A failed first question becomes a permanent belief.
Documentation was read by 12% before first use. It worked for them and the content belongs in the interface.
Relocate the document into the empty state, the starters, the failure screens, the citation panel and the feedback control — all of which already exist.
The first five minutes need one question that works, one citation opened, and one honest refusal.
Include a deliberately-refused starter. Repeat use was 79% among users who hit a refusal in session one, against 64% who did not.
Roll out in waves, and put the team whose work it absorbs third, not first. Their endorsement decides everyone else's trust.
Rewrite the starters from a real team's traffic before the full rollout. That took repeat use from 34% to 71%.
Use contextual hints shown once at the relevant moment. 94% see them; a first-login tour was dismissed in four seconds by 71%.
Do not demo the most impressive capability, do not run training sessions, and do not call it AI. Describing it as search over company documents halved first-week out-of-scope questions.

Next: setting expectations so trust survives the first wrong answer — because there will be one.