All chapters
  1. Start hereOne change, from diff to production
  2. Before stage 1The team setup: what changes when you are not alone
  3. Stage 1Code change: catch it while the agent is still editing
  4. Stage 2Commit: what must be true before a commit exists
  5. Stage 3Pull request: arrive already reviewed, with evidence
  6. Stage 4Review: who reviews what, at hundreds a day
  7. Stage 5CI/CD: spend scarce capacity like a budget
  8. Deep diveCI/CD agents: where they live and how they run
  9. Deep diveBuild the agents: pinpoint the change, then fix it
  10. Stage 6When the change is an agent
  11. OperateVolume and cost: what breaks at 300 a day
  12. AdoptAdopt it in steps
  13. ReferenceHook events: which one, with a script for each
  14. ReferenceGlossary
  15. AboutAbout this site
Adopt

Adopt it in steps

What to turn on first, what to measure before the next step, and the order that keeps trust.

Picture a team about Acme's size that decides to adopt all of this at once. The story is illustrative, and so are the durations on this page.

On Monday the reviewer agent starts commenting on every pull request. On Wednesday the lane rules go live, and green-lane changes merge with no person. On Friday the last piece is switched on: when main goes red, the triage agent's claim opens a revert and merges it automatically, so nobody waits for a person on call.

The next Tuesday main goes red. A sign-in test has timed out. The triage agent reads the log, sees seven changes since the last good build, and names the newest one, with high confidence. The revert merges minutes later.

Main stays red, because the test is flaky: it has an open flaky-test record, and it failed for no reason connected to any change. The developer whose change was reverted did nothing wrong.

By the afternoon the question is no longer about one revert. If the triage agent was wrong and sure, why trust the reviewer's comments? Who checked the lane rules that let green changes merge alone? Nobody can answer, because nobody measured anything before the week started, and within days all three switches are off.

The tools were not the mistake. The order was.

Add the check before the autonomy. At every step, first build the thing that tells you whether the agent was right, then let the agent do more.

1. Why one wrong revert turned off everything#

Three things went wrong that week, and each has a cause that an order of adoption can remove.

The first is that nothing was measured before. The team could not say whether review got faster, whether main was red for less time, or whether more defects escaped, because it had no "before" to compare with. Without a baseline, every incident is an argument, and the loudest one wins.

The second is that the autonomy came before the check. The revert acted on a model's claim that no code had tested. On the day of #4812, the same kind of claim went through two checks first.

A script compares the failure with open flaky-test records, and that alone would have stopped this one: the failure was already known, so nothing else starts. And the verifier, a CI job with no model, re-runs the failing test at the failing build, at the suspect change and at its parent; it reproduces first and repeats a flaky target, and a failure that does not repeat gives an inconclusive verdict, not a revert. That loop is on CI/CD.

The third is that every switch came in the same week, so no switch had a record of its own. When one failed, the others had nothing to defend them. A step that has run for a month with a known acceptance rate survives a bad day on another step; a step that started on Monday does not.

So the rule for adoption follows from the causes. Each step must be useful on its own, must produce a number, and must not give an agent more authority until a check for that authority exists and has been seen working.

2. The order#

The nine steps below follow that rule. Read the diagram left to right: each column adds one capability and one number to watch, and nothing in a later column is turned on until the earlier number looks right.

Diagram: Adopt in steps · a verifier before each new autonomy

Read left to right. Each column adds one capability and one number to watch. Nothing in a later column is turned on until the earlier number looks right.

  1. Measure and check
    1. Baseline: Review wait, end-to-end runs per merged change, escaped defects.
    2. Hooks in the repository: After-edit and stop hooks, committed, the same for everyone.
    3. Server gate: Protected paths and owners for gate files.
  2. Agent review
    1. Advisory review: The reviewer agent comments and blocks nothing.
    2. Lanes + audit: Lane rules run in shadow first. Then green merges automatically; a weekly sample checks the rules.
    3. Receipts: Local review for feedback; later, receipts signed by a trusted runner that CI can reuse.
  3. Capacity
    1. CI budget: Merge queue, tiers, scheduling classes for environments, retry budget.
    2. Checked fix loop: Triage in shadow, then the controlled re-run verifier, then revert pull requests a person merges.
  4. Agents as code
    1. Agent-change pipeline: The skill-lint, trigger-test, agent-smoke and agent-eval checks for every agent change; then candidate, shadow, activate.
StepTurn onMeasure before moving onTypical time
1. BaselineNothing new. Count pull requests, review wait, end-to-end runs per merged change, escaped defects.You have the four numbers1 week
2. Inner loopHooks after each edit and at the stop moment, committed to the repositoryShare of CI failures that a local check would have caught goes down1 to 2 weeks
3. Server gateProtected paths, code owners for gate files, required checksNo change to a gate file merges without its owner1 week
4. Agent review, advisoryThe reviewer agent, which has no edit tools, comments on every pull request. It blocks nothing.Finding acceptance rate above a level the team agrees on2 to 4 weeks
5. LanesLane rules run in shadow first: computed and audited weekly, while people still approve everything. Then green merges automatically, with a named owner and a tested rollback.Audit failures near zero for a month before the switch6 weeks
6. ReceiptsLocal review for feedback first. Later, if the numbers justify it, a trusted runner that signs receipts CI can reuse.Fix rounds per pull request go down; for signed receipts, a reuse rate that pays for the service2 weeks, then a separate project
7. CI budgetMerge queue, tiered pipeline, priority for end-to-end environments, agent retry budgetEnd-to-end runs per merged change goes down2 to 4 weeks
8. Checked fix loopTriage agent at the record-only level (its claims are stored, nothing acts on them), then the controlled re-run verifier, then revert pull requests that a person mergesShare of triage claims the verifier supports; time main stays red4 weeks or more
9. Agent-change pipelineThe agent checks skill-lint, trigger-test and agent-smoke on the pull request, and agent-eval (the full golden set on frozen snapshots) in the merge queueNo agent change merges without an eval resultOngoing

Two patterns run through the table. Every step that gives an agent authority is preceded by the step that can check it: the server gate (3) comes before the reviewer (4), the lane audit runs in shadow before green merges alone (5), and the verifier exists before a revert pull request is opened (8). And every step names its number before it starts, so the decision to move on is a reading, not an opinion.

Step 8 is itself several moves. The triage agent earns authority in levels: record only, then advice to the people on call, then a claim the verifier checks, then a revert pull request that a person approves (the levels). A level says what a claim of the active version may cause. Each move up is a red-lane change to the pipeline, and it passes the same lifecycle as any agent change: merge, deploy as a candidate, shadow, activate; shadow there is a moment in the life of a new version, not an authority level. At Acme the version with authority is named in .github/agent-versions.yml, a gate file owned by @acme/platform-leads.

3. The day of #4812, replayed against the order#

The best test of an order is a real day. Acme had all nine steps in place on the day of #4812. Go through that day again and ask, for each event, which step caught it, contained it or measured it, and what would have happened without that step.

TimeEventStepCaught, contained or measuredWithout that step
09:16Lint error F841 in create_order.py2. Inner loopCaught by the after-edit hook, in secondsThe agent builds four more files on the mistake; the first signal is a CI failure minutes later
09:29test_response_includes_delivery_note fails2. Inner loopCaught by the Stop hook before the agent finishedA pull request opens with a failing test and costs a CI run and a fix round
09:34The local review misses both findings6. Receipts, local firstContained: the local review is feedback, never the verdict. Its miss is run-to-run variation (illustrative).If the local review were trusted as the verdict, both findings would have reached main
09:35lane computes red: two red paths3 and 5Contained: the owners of api/openapi/** and db/migrations/** must approveA change to a public contract and a migration could merge with one approval
09:40agent-review posts two findings4. Advisory reviewCaught; both findings led to a change, so both count toward the acceptance rateFinding 1, the missing length check, waits for a person to notice it
10:20The API owner decides finding 2: the field is optional3. Server gateCaught by a person, because the owners file requires the API owner whatever the agent saysThe decision to break existing clients rests on whether the agent happened to notice
10:48Two owners approve; the policy app sets lane to success5. LanesMeasured: the same lanes let #4822 merge with no person and #4819 with one teammateEvery change waits for people, or none does
10:50 to 11:02The merge queue runs e2e-smoke for a group of five on one of 12 environments7. CI budgetContained: the scarce tier is spent once for five changesEvery change, and every agent retry, competes for an environment
11:46e2e-full job 4411 fails: a null note breaks the confirmation emailnone before mergeNot caught. Every pre-merge check passed. Contained by the deploy rule: the commit never deployed.Under a rule that deploys after the queue, customers without a note get no confirmation email
11:48 to 12:35The triage claim (0.82), then the verifier: supported8. Checked fix loopContained: a claim tested by code before anyone actedA guess acts on main, as in the opening story, or people search seven suspects by hand
12:36 to 12:58Revert #4840, approved by an API owner8 and 3Contained: 72 minutes red, measuredMain stays red until someone finds the cause
next dayA contract test that sends null is added1. Baseline, kept runningMeasured: one escaped defect becomes a cheaper check in integration-testsThe same mistake fails in the 40-minute run again
another day#4827 edits the triage agent's skill read-build-log9. Agent-change pipelineCaught: agent-smoke shows two cases moving from passing every run to failing every runThe edited agent blames the newest change, the opening story's mistake

Read the "Step" column from top to bottom. Each step did one job on that day, and no step did another step's job.

The inner loop never saw the red paths. The reviewer never had the authority to decide finding 2. The verifier never needed the reviewer.

One event went through every step: the null note that broke the confirmation email at 11:46. No check before merge could see it, for the reasons set out on CI/CD. What the order did was keep it small.

The deploy rule kept it away from customers (CI/CD, deploy), the checked fix loop kept main red for 72 minutes instead of an afternoon, and the next day's contract test turned it into a cheaper check (CI/CD, section 12). An order of adoption does not promise that nothing escapes. It promises that what escapes is contained, measured and turned into a check.

The 72 minutes is also the number step 1 exists for. Alone, it says nothing. Against a baseline of how long main stayed red before step 8, it says whether step 8 is worth what it costs.

4. Approaches weighed#

The opening story is one approach to adoption. There are others, and each buys something.

ApproachWhat it buysWhat goes wrong
Big bangA clear launch dateNothing is measured against a baseline. The first incident discredits everything at once.
One pilot team, all stepsFast learningThe pilot team's habits do not transfer. Other teams inherit rules they did not shape.
One step at a time, all teamsEach step has a before and afterSlower. Needs someone who owns the rollout.
Advisory before enforcingPeople see the agent be right before it can block themAdvisory comments are easy to ignore; set an end date

Recommended default: one step at a time, advisory before enforcing, with a named owner for the lane rules. It is the slowest of the four, and its slowness is the point: each step's number has time to become a record that the next step can lean on.

5. What stays with people#

Even at step 9, some decisions do not move to an agent, however good its numbers become. On the day of #4812, each of them was made by a person: the API owner on finding 2, the owners on the approvals, the API owner on the revert.

DecisionWhy it stays
What is wantedAn agent can check a change against the intent. It cannot own the intent.
Red-lane approvalAccountability for sign-in, payments and data-migration paths cannot be delegated to a tool.
The lane rulesWhoever sets the rules is accountable for what the green lane lets through.
RollbackA decision under uncertainty, with consequences outside the repository. An agent can prepare the rollback pull request and the evidence.
Changing an agent's authorityMoving an agent from advisory to acting is itself a red-lane change: a change to .github/agent-versions.yml or to the agent's files, with the platform owners' approval.

6. Signs it is going wrong#

An adoption rarely fails in one incident, as in the opening story. More often it drifts. Each sign below has a likely cause, and most of them point back to a step that was skipped or rushed.

SignLikely cause
People approve faster and read lessApproach B on the Review page: an agent summary replaced the reading
Agent comments are ignoredToo many findings, not verified before posting
The end-to-end queue is longer than beforeNo retry budget; no tiering
Developers hide agent useLanes or blame assigned by author
An agent change "looked fine" and broke triageText was reviewed; behavior was not

7. Side questions#

QuestionShort answer
Where do people who are new to coding agents start?Step 2, on their own machine, with a first task that only reads. The committed hooks and project instructions mean a new person gets the team's shared setup from the repository, alongside their own personal settings.
Small team, no scarce test environments?Steps 2 to 5 are most of the value. Skip the end-to-end budget; keep the lanes.
Regulated environment?The lanes and the evidence on each pull request map well to change control. Separation of duties holds: the author, person or agent, never approves.

Go back to the team in the opening story, and give it the same tools in this order. The triage agent would have spent its first month at the record-only level, its claims stored and compared with what happened. The flaky sign-in test would have matched its open record before any agent ran, and if it had not, the verifier would most likely have returned inconclusive, because a flaky failure rarely repeats in every run. The wrong revert would have been one line in a log, not an incident.

Trust in an agent is not a feature that can be switched on; it is a record of the checker having been right, and the order of adoption is how a team builds that record one number at a time.

Next: the hooks behind step 2, the event-by-event reference for the inner loop, on Hooks.