Picture a team about Acme's size that decides to adopt all of this at once. The story is illustrative, and so are the durations on this page.
On Monday the reviewer agent starts commenting on every pull request. On Wednesday the lane rules go live, and green-lane changes merge with no person. On Friday the last piece is switched on: when main goes red, the triage agent's claim opens a revert and merges it automatically, so nobody waits for a person on call.
The next Tuesday main goes red. A sign-in test has timed out. The triage agent reads the log, sees seven changes since the last good build, and names the newest one, with high confidence. The revert merges minutes later.
Main stays red, because the test is flaky: it has an open flaky-test record, and it failed for no reason connected to any change. The developer whose change was reverted did nothing wrong.
By the afternoon the question is no longer about one revert. If the triage agent was wrong and sure, why trust the reviewer's comments? Who checked the lane rules that let green changes merge alone? Nobody can answer, because nobody measured anything before the week started, and within days all three switches are off.
The tools were not the mistake. The order was.
Add the check before the autonomy. At every step, first build the thing that tells you whether the agent was right, then let the agent do more.
1. Why one wrong revert turned off everything#
Three things went wrong that week, and each has a cause that an order of adoption can remove.
The first is that nothing was measured before. The team could not say whether review got faster, whether main was red for less time, or whether more defects escaped, because it had no "before" to compare with. Without a baseline, every incident is an argument, and the loudest one wins.
The second is that the autonomy came before the check. The revert acted on a model's claim that no code had tested. On the day of #4812, the same kind of claim went through two checks first.
A script compares the failure with open flaky-test records, and that alone would have stopped this one: the failure was already known, so nothing else starts. And the verifier, a CI job with no model, re-runs the failing test at the failing build, at the suspect change and at its parent; it reproduces first and repeats a flaky target, and a failure that does not repeat gives an inconclusive verdict, not a revert. That loop is on CI/CD.
The third is that every switch came in the same week, so no switch had a record of its own. When one failed, the others had nothing to defend them. A step that has run for a month with a known acceptance rate survives a bad day on another step; a step that started on Monday does not.
So the rule for adoption follows from the causes. Each step must be useful on its own, must produce a number, and must not give an agent more authority until a check for that authority exists and has been seen working.
2. The order#
The nine steps below follow that rule. Read the diagram left to right: each column adds one capability and one number to watch, and nothing in a later column is turned on until the earlier number looks right.
Diagram: Adopt in steps · a verifier before each new autonomy
Read left to right. Each column adds one capability and one number to watch. Nothing in a later column is turned on until the earlier number looks right.
- Measure and check
- Baseline: Review wait, end-to-end runs per merged change, escaped defects.
- Hooks in the repository: After-edit and stop hooks, committed, the same for everyone.
- Server gate: Protected paths and owners for gate files.
- Agent review
- Advisory review: The reviewer agent comments and blocks nothing.
- Lanes + audit: Lane rules run in shadow first. Then green merges automatically; a weekly sample checks the rules.
- Receipts: Local review for feedback; later, receipts signed by a trusted runner that CI can reuse.
- Capacity
- CI budget: Merge queue, tiers, scheduling classes for environments, retry budget.
- Checked fix loop: Triage in shadow, then the controlled re-run verifier, then revert pull requests a person merges.
- Agents as code
- Agent-change pipeline: The skill-lint, trigger-test, agent-smoke and agent-eval checks for every agent change; then candidate, shadow, activate.
| Step | Turn on | Measure before moving on | Typical time |
|---|---|---|---|
| 1. Baseline | Nothing new. Count pull requests, review wait, end-to-end runs per merged change, escaped defects. | You have the four numbers | 1 week |
| 2. Inner loop | Hooks after each edit and at the stop moment, committed to the repository | Share of CI failures that a local check would have caught goes down | 1 to 2 weeks |
| 3. Server gate | Protected paths, code owners for gate files, required checks | No change to a gate file merges without its owner | 1 week |
| 4. Agent review, advisory | The reviewer agent, which has no edit tools, comments on every pull request. It blocks nothing. | Finding acceptance rate above a level the team agrees on | 2 to 4 weeks |
| 5. Lanes | Lane rules run in shadow first: computed and audited weekly, while people still approve everything. Then green merges automatically, with a named owner and a tested rollback. | Audit failures near zero for a month before the switch | 6 weeks |
| 6. Receipts | Local review for feedback first. Later, if the numbers justify it, a trusted runner that signs receipts CI can reuse. | Fix rounds per pull request go down; for signed receipts, a reuse rate that pays for the service | 2 weeks, then a separate project |
| 7. CI budget | Merge queue, tiered pipeline, priority for end-to-end environments, agent retry budget | End-to-end runs per merged change goes down | 2 to 4 weeks |
| 8. Checked fix loop | Triage agent at the record-only level (its claims are stored, nothing acts on them), then the controlled re-run verifier, then revert pull requests that a person merges | Share of triage claims the verifier supports; time main stays red | 4 weeks or more |
| 9. Agent-change pipeline | The agent checks skill-lint, trigger-test and agent-smoke on the pull request, and agent-eval (the full golden set on frozen snapshots) in the merge queue | No agent change merges without an eval result | Ongoing |
Two patterns run through the table. Every step that gives an agent authority is preceded by the step that can check it: the server gate (3) comes before the reviewer (4), the lane audit runs in shadow before green merges alone (5), and the verifier exists before a revert pull request is opened (8). And every step names its number before it starts, so the decision to move on is a reading, not an opinion.
Step 8 is itself several moves. The triage agent earns authority in levels: record only, then advice to the people on call, then a claim the verifier checks, then a revert pull request that a person approves (the levels). A level says what a claim of the active version may cause. Each move up is a red-lane change to the pipeline, and it passes the same lifecycle as any agent change: merge, deploy as a candidate, shadow, activate; shadow there is a moment in the life of a new version, not an authority level. At Acme the version with authority is named in .github/agent-versions.yml, a gate file owned by @acme/platform-leads.
3. The day of #4812, replayed against the order#
The best test of an order is a real day. Acme had all nine steps in place on the day of #4812. Go through that day again and ask, for each event, which step caught it, contained it or measured it, and what would have happened without that step.
| Time | Event | Step | Caught, contained or measured | Without that step |
|---|---|---|---|---|
| 09:16 | Lint error F841 in create_order.py | 2. Inner loop | Caught by the after-edit hook, in seconds | The agent builds four more files on the mistake; the first signal is a CI failure minutes later |
| 09:29 | test_response_includes_delivery_note fails | 2. Inner loop | Caught by the Stop hook before the agent finished | A pull request opens with a failing test and costs a CI run and a fix round |
| 09:34 | The local review misses both findings | 6. Receipts, local first | Contained: the local review is feedback, never the verdict. Its miss is run-to-run variation (illustrative). | If the local review were trusted as the verdict, both findings would have reached main |
| 09:35 | lane computes red: two red paths | 3 and 5 | Contained: the owners of api/openapi/** and db/migrations/** must approve | A change to a public contract and a migration could merge with one approval |
| 09:40 | agent-review posts two findings | 4. Advisory review | Caught; both findings led to a change, so both count toward the acceptance rate | Finding 1, the missing length check, waits for a person to notice it |
| 10:20 | The API owner decides finding 2: the field is optional | 3. Server gate | Caught by a person, because the owners file requires the API owner whatever the agent says | The decision to break existing clients rests on whether the agent happened to notice |
| 10:48 | Two owners approve; the policy app sets lane to success | 5. Lanes | Measured: the same lanes let #4822 merge with no person and #4819 with one teammate | Every change waits for people, or none does |
| 10:50 to 11:02 | The merge queue runs e2e-smoke for a group of five on one of 12 environments | 7. CI budget | Contained: the scarce tier is spent once for five changes | Every change, and every agent retry, competes for an environment |
| 11:46 | e2e-full job 4411 fails: a null note breaks the confirmation email | none before merge | Not caught. Every pre-merge check passed. Contained by the deploy rule: the commit never deployed. | Under a rule that deploys after the queue, customers without a note get no confirmation email |
| 11:48 to 12:35 | The triage claim (0.82), then the verifier: supported | 8. Checked fix loop | Contained: a claim tested by code before anyone acted | A guess acts on main, as in the opening story, or people search seven suspects by hand |
| 12:36 to 12:58 | Revert #4840, approved by an API owner | 8 and 3 | Contained: 72 minutes red, measured | Main stays red until someone finds the cause |
| next day | A contract test that sends null is added | 1. Baseline, kept running | Measured: one escaped defect becomes a cheaper check in integration-tests | The same mistake fails in the 40-minute run again |
| another day | #4827 edits the triage agent's skill read-build-log | 9. Agent-change pipeline | Caught: agent-smoke shows two cases moving from passing every run to failing every run | The edited agent blames the newest change, the opening story's mistake |
Read the "Step" column from top to bottom. Each step did one job on that day, and no step did another step's job.
The inner loop never saw the red paths. The reviewer never had the authority to decide finding 2. The verifier never needed the reviewer.
One event went through every step: the null note that broke the confirmation email at 11:46. No check before merge could see it, for the reasons set out on CI/CD. What the order did was keep it small.
The deploy rule kept it away from customers (CI/CD, deploy), the checked fix loop kept main red for 72 minutes instead of an afternoon, and the next day's contract test turned it into a cheaper check (CI/CD, section 12). An order of adoption does not promise that nothing escapes. It promises that what escapes is contained, measured and turned into a check.
The 72 minutes is also the number step 1 exists for. Alone, it says nothing. Against a baseline of how long main stayed red before step 8, it says whether step 8 is worth what it costs.
4. Approaches weighed#
The opening story is one approach to adoption. There are others, and each buys something.
| Approach | What it buys | What goes wrong |
|---|---|---|
| Big bang | A clear launch date | Nothing is measured against a baseline. The first incident discredits everything at once. |
| One pilot team, all steps | Fast learning | The pilot team's habits do not transfer. Other teams inherit rules they did not shape. |
| One step at a time, all teams | Each step has a before and after | Slower. Needs someone who owns the rollout. |
| Advisory before enforcing | People see the agent be right before it can block them | Advisory comments are easy to ignore; set an end date |
Recommended default: one step at a time, advisory before enforcing, with a named owner for the lane rules. It is the slowest of the four, and its slowness is the point: each step's number has time to become a record that the next step can lean on.
5. What stays with people#
Even at step 9, some decisions do not move to an agent, however good its numbers become. On the day of #4812, each of them was made by a person: the API owner on finding 2, the owners on the approvals, the API owner on the revert.
| Decision | Why it stays |
|---|---|
| What is wanted | An agent can check a change against the intent. It cannot own the intent. |
| Red-lane approval | Accountability for sign-in, payments and data-migration paths cannot be delegated to a tool. |
| The lane rules | Whoever sets the rules is accountable for what the green lane lets through. |
| Rollback | A decision under uncertainty, with consequences outside the repository. An agent can prepare the rollback pull request and the evidence. |
| Changing an agent's authority | Moving an agent from advisory to acting is itself a red-lane change: a change to .github/agent-versions.yml or to the agent's files, with the platform owners' approval. |
6. Signs it is going wrong#
An adoption rarely fails in one incident, as in the opening story. More often it drifts. Each sign below has a likely cause, and most of them point back to a step that was skipped or rushed.
| Sign | Likely cause |
|---|---|
| People approve faster and read less | Approach B on the Review page: an agent summary replaced the reading |
| Agent comments are ignored | Too many findings, not verified before posting |
| The end-to-end queue is longer than before | No retry budget; no tiering |
| Developers hide agent use | Lanes or blame assigned by author |
| An agent change "looked fine" and broke triage | Text was reviewed; behavior was not |
7. Side questions#
| Question | Short answer |
|---|---|
| Where do people who are new to coding agents start? | Step 2, on their own machine, with a first task that only reads. The committed hooks and project instructions mean a new person gets the team's shared setup from the repository, alongside their own personal settings. |
| Small team, no scarce test environments? | Steps 2 to 5 are most of the value. Skip the end-to-end budget; keep the lanes. |
| Regulated environment? | The lanes and the evidence on each pull request map well to change control. Separation of duties holds: the author, person or agent, never approves. |
Go back to the team in the opening story, and give it the same tools in this order. The triage agent would have spent its first month at the record-only level, its claims stored and compared with what happened. The flaky sign-in test would have matched its open record before any agent ran, and if it had not, the verifier would most likely have returned inconclusive, because a flaky failure rarely repeats in every run. The wrong revert would have been one line in a log, not an incident.
Trust in an agent is not a feature that can be switched on; it is a record of the checker having been right, and the order of adoption is how a team builds that record one number at a time.
Next: the hooks behind step 2, the event-by-event reference for the inner loop, on Hooks.