All chapters
  1. Start hereOne change, from diff to production
  2. Before stage 1The team setup: what changes when you are not alone
  3. Stage 1Code change: catch it while the agent is still editing
  4. Stage 2Commit: what must be true before a commit exists
  5. Stage 3Pull request: arrive already reviewed, with evidence
  6. Stage 4Review: who reviews what, at hundreds a day
  7. Stage 5CI/CD: spend scarce capacity like a budget
  8. Deep diveCI/CD agents: where they live and how they run
  9. Deep diveBuild the agents: pinpoint the change, then fix it
  10. Stage 6When the change is an agent
  11. OperateVolume and cost: what breaks at 300 a day
  12. AdoptAdopt it in steps
  13. ReferenceHook events: which one, with a script for each
  14. ReferenceGlossary
  15. AboutAbout this site
Deep dive

CI/CD agents: where they live and how they run

Skills and agent definitions are the same files locally and in a pipeline. Where each is stored, what a CI run loads, how a workflow starts a run, and the trap of a pull request that edits its own reviewer.

Two agents worked on #4812 with nobody watching: the reviewer that posted both findings at 09:40 (Review), and the triage agent that named #4812 at 11:55, after main went red (CI/CD). Those pages showed what each agent did, not where it came from. Each ran on a runner that had never seen anyone's laptop, so each had to be defined in files the runner could reach and started by a workflow. Where those files live decides what a run loads, and an agent's rules have to live at the right level.

The examples use Claude Code and GitHub Actions. Other coding agents and other hosts use different names for the same parts. Product behavior was checked against the vendor's documentation on the date in the footer.

Organization rules belong in managed settings. Team rules belong in the repository. The personal folder is for personal preferences only. A pipeline sees the first two and never the third.

The standard: three levels, and what belongs at each#

LevelWhere it livesWho it applies toWhat belongs thereWhat must not be there
OrganizationManaged settings, deployed by an administratorEvery developer, and every runner the organization installs them onRules that must hold for everyone and that nobody may loosen: security limits, approved tools, compliance rulesTeam-specific detail that changes often
TeamThe repository: CLAUDE.md, .claude/, and folders for pipeline agentsEveryone who works in that repository, and every pipeline run on itThe team's review rules, skills, agent definitions, hooks and permissions. Reviewed and owned like code.Secrets. Personal taste.
PersonalThe developer's home folderOne person, on one machinePersonal preferences: an editor habit, a private shortcutAnything the team or the pipeline depends on

Two rules follow, and the rest of the page depends on them.

What goes wrong when the standard is not followed#

Before the review rules were in Acme's repository, well before the day of #4812, a developer wrote them as a skill. Her coding agent follows them: it reports only what it can support with a line of code, and it marks any change to a public API contract as "needs the API owner". The rules are good. Their location is the mistake: she saved them in her personal folder, so the team's rules exist on one laptop.

The team wants that same review on every pull request. She adds a workflow file that starts the agent in CI (continuous integration, the system that runs checks on every change). The first run finishes without an error. But the review reads like a stranger wrote it. It comments on formatting. It says nothing about the API owner.

Nothing is broken. The agent in CI did what it was told. It was told far less than the agent on her laptop. The story is illustrative, and the next section explains exactly why it happens; by the day of #4812 the rules were in the repository, which is why the 09:40 review found both findings.

1. Why the review came back generic#

On her laptop, the agent gets its instructions from two places. Some come from the repository she is working in. Others come from her home folder, the personal folder on her own machine. She had saved the review rules there, against the standard above, so that they would apply in every project she opens.

A CI job does not run on her laptop. It runs on a runner: a clean machine that the CI system starts for one job and throws away afterward. The runner does one thing before the agent starts. It copies the repository onto its disk. That copy is called the checkout.

So the agent in CI sees the repository and nothing else. Her home folder is not there. The review rules were not in the repository, so for that run they did not exist.

The agent's instructions come fromOn her laptopOn the runner
Files in the repositoryLoadedLoaded, from the checkout
Files in her home folderLoadedNot there. The runner has no home folder of hers.
What she typesYesNobody types. The workflow supplies the prompt.

This is the whole difference, and it explains the generic review. It also answers a question many people ask first: are skills and agents defined differently for a pipeline? They are not. There is no second format. The same files, in the same folders, work in both places. The only requirement is that the pipeline can reach them.

2. The repository becomes the agent's home#

The fix is to follow the standard. Team rules belong at the team level, so she moves the review rules from her home folder into the repository, where the team can review and own them. From then on every run gets them: hers, her colleagues', and the runner's.

Claude Code looks for specific files in specific places inside a repository. Two of them matter most here, and they solve two different problems.

A skill answers "how should this task be done?". It is a file of instructions for one kind of task. The review rules are a skill.

A skill file.claude/skills/review-rules/SKILL.md

The file has a short header and then the instructions in plain language.

---
name: review-rules
description: The team's rules for reviewing a pull request. Use for any code review.
---

Report only findings you can support with a line of code.
1. Correctness first: a wrong result, a missing check, an unhandled empty value.
2. A change to a public contract in api/openapi/ is never a style issue. Mark it "needs the API owner".
3. Tests: every new branch in the code has a test. A removed or skipped test needs a reason.
4. At most five findings. Most serious first.
Do not comment on formatting. The formatter handles it.

Rule 2 and the last line are exactly what the first CI run lacked.

One detail of how skills load explains a common surprise. At the start of a run, the model sees only each skill's name and description. The body is loaded when the model decides to use the skill, and it decides from the description. So a skill with a vague description may never be used, however good its body is.

Fact about skills in Claude CodeWhat it means in practice
Only the name and description are in the context at the startMany skills cost little until one is needed
The model picks a skill from its descriptionWrite when to use the skill, not only what it is
A prompt can name a skillIn a run with no person, a prompt that starts with a slash and the skill name runs that skill. This is how a workflow invokes one.
The folder name is the command nameThe folder .claude/skills/ship/ gives the command ship

The skill fixes the content of the review. A second problem remains. A reviewer should read code and comment on it. It should not be able to change the code it is judging. Instructions cannot guarantee that; an instruction is advice, and the agent still holds its editing tools.

A subagent answers "who does this task, and with which powers?". It is a separate agent with its own instructions, its own list of tools and its own limits. A tool that is not on its list is not available to it, whatever a prompt or a pull request says.

A subagent file.claude/agents/code-reviewer.md

The header sets the powers. The text under it is the subagent's own system prompt.

---
name: code-reviewer
description: Reviews a diff for correctness bugs. Has no editing tools. Use for every pull request review.
tools: Read, Grep, Glob, Bash
model: sonnet
maxTurns: 15
skills:
  - review-rules
---

You review one pull request. You cannot edit files.
Read the diff against the base branch, then the files around each change.
Report findings as: file, line, severity, one sentence on what is wrong, one on the fix.
End with one line: VERDICT: pass  or  VERDICT: changes-needed

The line that matters most is tools. It is a list of whole tools: reading, searching and the shell. The file-editing tools are not on it, so the reviewer has no edit or write tool.

That does not make it read-only. The shell is on the list, because the reviewer needs git to see the diff, and a shell command can also change a file. The tools field names whole tools; it does not narrow the shell to certain commands. That narrowing is done where the run is started, in the workflow in section 3, and even there it has limits.

Header fieldWhat it does
nameHow the agent is called. Required.
descriptionTells the main agent when to hand work to this one. Required.
toolsThe whole tools this agent may use. To limit which shell commands are allowed, use permission rules or the start command, not this field.
modelWhich model it runs on
maxTurnsA cap on its steps
skillsSkills loaded in full when it starts. Here: the review rules.
SkillSubagent
The question it answersHow is this task done?Who does it, and what may they touch?
What it isInstructions added to the agent that is already runningA separate agent with its own context, tools and limits
Its toolsThe tools the running agent already hasIts own list: the place to remove the editing tools

Both now sit in the repository, next to the code they apply to. One more kind of file belongs there too. A tool server is a separate program that offers tools to the agent: one that reads CI logs, say, or searches a bug tracker. The agent finds and calls those tools through MCP, the Model Context Protocol, the common standard by which an agent discovers and calls tools from a tool server. In Claude Code, a tool from a server named triage is called mcp__triage__<tool>, and that full name is what permission rules list.

Here is where each kind of file lives. The last two rows are the ones the opening story turned on.

WhatFileOn a developer machineIn a CI run
Project instructionsCLAUDE.md at the repository rootLoadedLoaded from the checkout
Settings: permissions, deny rules, hooks.claude/settings.jsonLoadedLoaded from the checkout
A skill.claude/skills/<name>/SKILL.mdLoadedLoaded from the checkout
A subagent.claude/agents/<name>.mdLoadedLoaded from the checkout
Tool servers.mcp.json at the repository rootLoaded after the developer approves themLoaded from the checkout
Personal skills, agents and settingsThe same file names, under the home folderLoadedNot there
The workflow.github/workflows/<name>.ymlNot usedThis starts the run

When the same name exists in two places#

Moving the rules is half of the correction. The developer still has her old copy in her home folder, under the same name, review-rules. On her laptop both copies are now present. Which one does her agent use?

It is tempting to answer "the repository wins". For some kinds of file that is true. For skills it is the opposite, and the difference matters, because her personal copy is the old one.

Claude Code's documentation gives a separate rule for each kind of file.

Kind of fileWhen the same name or key exists at several levelsDoes the repository beat the home folder?
Settings (settings.json: permissions, hooks)Highest first: organization-managed, command line, the project's local file, the project's shared file, the personal fileYes
SubagentsHighest first: organization-managed, the command-line flag, the project folder, the personal folder, a pluginYes
SkillsOrganization over personal, and personal over projectNo. The personal copy runs.
Instruction files (CLAUDE.md)All of them are loaded. Personal first, project after. Neither overrides the other; if two conflict, the agent may follow either.Neither wins

So on her laptop, the agent would keep using the old personal skill and ignore the one the team now maintains. In CI the problem cannot occur, because the runner has no home folder: only the repository's copy exists there. The same developer would then see one review locally and a different one on the pull request. She must delete the personal copy. That completes the correction: the team's rules now exist in one place, at the team level.

This is also why a team cannot rely on "we committed it, so everyone uses it". A file in the repository is shared. It is not enforced. Another developer may have a personal skill with the same name, and on that machine it would win.

What the organization must do for rules that cannot vary#

For a rule that has to hold for every developer, the repository is not enough, as the table above shows. The organization must use the level above both the home folder and the repository: managed settings. An administrator deploys them to each machine, or from the vendor's admin console. A developer cannot override them from a personal or a project file. For a pipeline, the organization installs them on the runner as well, or keeps the rule in the repository where the runner can read it.

The organization wantsThe managed controlWhat it does
One rule that nobody can loosenA key in managed settingsManaged settings sit at the top of the order. No project or personal file overrides them.
A skill or a subagent that is the same for everyoneAn organization-level skill or subagent, deployed with the managed settingsIt wins over a personal or project one with the same name
No personal or project customization at allThe managed setting strictPluginOnlyCustomizationBlocks skills, subagents, hooks and tool servers that come from personal and project folders. Only approved plugins supply them. It can lock all four kinds or a chosen subset.
Company instructions in every sessionA managed instruction fileLoaded for all users, before personal and project instructions
Only approved hooks and permission rulesThe managed settings allowManagedHooksOnly and allowManagedPermissionRulesOnlyHooks and permission rules from other levels are ignored

Two limits keep this honest. Managed settings govern the agent, not the developer: a person can still edit code by hand outside the agent. And instruction files guide the model; they do not block an action. A rule that must hold belongs in a setting, a hook or a server-side check, not in a sentence of instructions.

3. What the workflow file adds#

The repository now holds everything the reviewer needs. But a file does nothing by itself. Something has to start the agent at the right moment, on the right code, with the right permissions. That is the job of the workflow file, and it is the only part of this setup that is specific to GitHub.

A workflow file answers three questions. Which event starts the run? What may the job touch? Which command starts the agent?

The reviewer's workflowIt names the subagent. The repository defines it.

This workflow starts when a pull request opens or receives a new commit. It gives the job permission to read the code, to write comments, and to request the identity token that the Action signs in with. Then it starts the agent as the code-reviewer subagent, and it narrows the shell to two git subcommands. This is Acme's one agent-review.yml; Review links here for it.

# .github/workflows/agent-review.yml
name: agent-review
on:
  pull_request:
    types: [opened, synchronize, ready_for_review, reopened]
  merge_group:                      # a required check must also report in the merge queue
permissions:
  contents: read
  pull-requests: write
  id-token: write                   # required by the Action's default GitHub App sign-in
concurrency:                        # a new commit cancels the review of the old one
  group: agent-review-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true
jobs:
  agent-review:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - if: ${{ github.event_name == 'merge_group' }}
        run: echo "Reviewed on the pull request; nothing to review in the queue."
      - if: ${{ github.event_name == 'pull_request' }}
        uses: actions/checkout@v6
        with: { fetch-depth: 0 }
      - if: ${{ github.event_name == 'pull_request' }}
        uses: anthropics/claude-code-action@v1
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          prompt: "Review the changes in this pull request against its base branch."
          claude_args: >-
            --agent code-reviewer
            --allowedTools "Read,Grep,Glob,Bash(git diff *),Bash(git log *),mcp__github_inline_comment__create_inline_comment"
            --permission-mode dontAsk
            --max-turns 15 --max-budget-usd 2

Notice how little the workflow says about the review itself. It does not contain the rules or the tool list. It points at them.

Line in the workflowWhere the thing it names is defined
--agent code-reviewer.claude/agents/code-reviewer.md in the checkout. The whole run takes that agent's tools, model and prompt.
--allowedTools with --permission-mode dontAskNot a file. The first flag approves reading and searching, two git subcommands, and the Action's tool for posting a comment on a line. The second refuses everything that was not approved, without asking. Together they keep the reviewer to reading and commenting. They are permission rules on command text, not a sandbox: an approved subcommand still accepts every option, and both git diff and git log have an option that writes output to a file. So the reviewer also gets nothing worth protecting: no secret beyond the model key, and a token that can only read code and write comments.
The skill review-rules.claude/skills/review-rules/SKILL.md, loaded because the subagent's header lists it
anthropic_api_keyA secret stored in the CI system. Never a file in the repository. This is the simpler start; the credentials standard is in section 8.
permissionsWhat the job's GitHub token may do. id-token: write stays because the Action's documentation requires it for its default GitHub App sign-in, the identity whose comments and commits it posts.
merge_group and the first stepagent-review is a required check, so it must also report in the merge queue, or each queued entry waits until the queue times out. In the queue the job runs only the first step and reports success at once: the review already happened on the pull request. The job keeps one name, so the check is the same check in both places.
concurrencyA new commit on the same pull request cancels the review still running on the old one, instead of paying for both. In the queue there is no pull request number, so each queue branch gets its own group and no queue run cancels another.
claude_argsAny command-line flag of the agent can go here. This combination is assembled from documented flags; try it on a test repository first.

One honest note about where the rules come from. On the pull_request event, the checkout is the pull request's own merge commit, so this workflow loads the subagent, the skill and CLAUDE.md from the pull request it reviews. Its protection is that those files are gate files with an owner. Loading them from the base branch is stronger; section 7 describes it as a design.

With that file in place, the next review is a different review. It skips formatting. On a change like #4812 it reports the missing length check that the contract promises, and it marks the required delivery_note field "needs the API owner", because rule 2 of the skill says so. Those are the two findings of 09:40.

4. An agent that only the pipeline uses#

The reviewer is useful in both places. A developer runs it before pushing; CI runs it on the pull request. One set of files serves both, and that is why they sit under .claude/.

Acme has a second agent for which that reasoning fails. When a test run on the main branch fails, a triage agent reads the logs and names the change that most likely broke it. No developer runs it by hand. It exists only for the pipeline.

Where should its files go? The obvious answer is the same folder, .claude/. The trouble is what that folder means: every file under it is loaded into every session in the repository. Each developer's coding agent would carry the triage agent's skills in its context, where they are useless and make the list of skills longer and less clear.

So the triage agent gets a folder of its own, outside .claude/. That solves the noise and creates one new duty. Claude Code does not search that folder by itself, so the command that starts the run must name it.

The folders, in one repositoryWhat every session loads, and what only the pipeline loads
acme/
├── CLAUDE.md                         project instructions: read by every run, local and CI
├── .mcp.json                         tool servers for the coding agent
├── .claude/
│   ├── settings.json                 permissions, deny rules, hooks
│   ├── hooks/                        scripts the hooks run
│   ├── skills/
│   │   ├── ship/SKILL.md             "push the branch and open the pull request"
│   │   └── review-rules/SKILL.md     the team's review checklist
│   └── agents/
│       ├── code-reviewer.md          a reviewer with no edit tools
│       └── fix-finding.md            the fix-on-request agent: may edit; fixes one named finding
├── agents/
│   ├── triage/                       a pipeline agent, packaged as a plugin folder
│   │   ├── .claude-plugin/plugin.json
│   │   ├── skills/read-build-log/SKILL.md
│   │   ├── skills/search-bug-history/SKILL.md
│   │   ├── skills/handle-tool-error/SKILL.md
│   │   ├── prompts/system.md
│   │   ├── mcp.json                  its three read-only tools
│   │   ├── claim.schema.json
│   │   └── evals/
│   └── fix/                          the red-build fix agent: runs only after a verified claim
│       └── prompts/task.md
└── .github/
    ├── agent-versions.yml            which version of each pipeline agent is active
    └── workflows/
        ├── agent-review.yml          starts the reviewer on a pull request
        ├── agent-fix.yml             starts the fix-on-request agent when a person asks in a comment
        └── red-build.yml             starts the triage agent when main fails

The folder agents/triage/ is laid out as a plugin: a folder that bundles skills, agent definitions and tool configuration so that they can be loaded together.

The triage agent's workflow and commandEvery file it needs is named on the command line

The workflow starts when the test workflow finishes on the main branch, and only if it failed. This job does not use the GitHub Action. It installs the agent and runs it as a plain command, in headless mode: one prompt goes in, one result comes out, and no person is present.

# .github/workflows/red-build.yml  (the triage job)
on:
  workflow_run:
    workflows: [Run Tests]
    types: [completed]
    branches: [main]
jobs:
  triage:
    if: ${{ github.event.workflow_run.conclusion == 'failure' }}
    runs-on: ubuntu-latest
    timeout-minutes: 20
    permissions: { contents: read, actions: read }
    steps:
      - uses: actions/checkout@v6
      - run: curl -fsSL https://claude.ai/install.sh | bash
      - env: { ANTHROPIC_API_KEY: "${{ secrets.ANTHROPIC_API_KEY }}", GH_TOKEN: "${{ github.token }}" }
        run: ci/triage.sh "${{ github.event.workflow_run.id }}"

The script takes one argument, the workflow run id (not a job id: the run contains the failing job). It does three things: it collects the input, it starts the agent, and it keeps the claim. Each piece the run needs is named on the command line.

#!/usr/bin/env bash
# ci/triage.sh RUN_ID : name the likely culprit of a red build on main
set -euo pipefail
run_id="$1"

ci/triage-input.sh "$run_id"      # writes input.json: failing job, failing test, failing and last passing build

claude -p "$(cat input.json)" \
  --plugin-dir agents/triage \
  --append-system-prompt "$(cat agents/triage/prompts/system.md)" \
  --mcp-config agents/triage/mcp.json --strict-mcp-config \
  --tools "Skill" \
  --allowedTools "Skill,mcp__triage__get_build_log,mcp__triage__list_changes,mcp__triage__search_bugs" \
  --permission-mode dontAsk --max-turns 25 --max-budget-usd 4 \
  --output-format json --json-schema "$(cat agents/triage/claim.schema.json)" > run.json

jq -e '.structured_output' run.json > claim.json   # no claim: exits non-zero, and the pipeline counts it as "unknown"

How ci/triage-input.sh finds the failing job and the last passing build is on Build the agents, section 5. Read the flags as a list of what would otherwise be missing, or unlimited.

FlagWhat it brings in or limits
--plugin-dir agents/triageThe skills under agents/triage/skills/. Without this flag they are not found.
--append-system-promptagents/triage/prompts/system.md
--mcp-config with --strict-mcp-configOnly the tool server in agents/triage/mcp.json. The project's own .mcp.json is not used. The file is named mcp.json, not .mcp.json, and the plugin manifest does not list it, so only this flag loads it and its tools keep the names mcp__triage__.... A server bundled inside a plugin gets the plugin's name in its tool names, and the allow list below would no longer match.
--tools "Skill"Nothing from a file. It removes every other built-in tool: no shell, no file read or edit. Without it, the agent would still have the built-in file and shell tools.
--allowedToolsNothing from a file. It approves the Skill tool and the three tools of the triage server. With --permission-mode dontAsk, anything else is refused. The list is in the command, so a reviewer of the workflow can see it.
--max-turns 25Nothing from a file. It stops the run after 25 turns.
--max-budget-usd 4Nothing from a file. It stops the run once the estimated spend reaches 4 USD. The estimate is Claude Code's own, calculated on the runner, and can differ from the bill; spend can also pass the cap before the run stops. Treat 4 USD as a target with headroom, and keep the job timeout of 20 minutes as the third cap.
--output-format json with --json-schemaagents/triage/claim.schema.json. The result is JSON, and its structured_output field holds the claim in that shape. A run that ends without valid structured output has no such field, which is why the last line checks for it.

The command is assembled from documented flags and has not been run against a real pipeline. It shows one version of the agent; how the workflow picks the active version from .github/agent-versions.yml is left out. Treat it as a starting point.

The rule that separates the two homes is short. If developers use the agent or the skill themselves, it goes under .claude/. If only the pipeline uses it, it gets its own folder, and its start command names that folder. How this triage agent is designed and tested is the subject of the next page.

5. Four ways to start an agent in CI#

The two examples used two different ways to start a run: an Action for the reviewer, a plain command for the triage agent. There are four ways in all. They differ in how much you build yourself.

WayWhat the workflow containsFitsThe cost
A. The agent's GitHub ActionA step that uses anthropics/claude-code-action@v1, with a prompt and flags in claude_argsReview on a pull request; answering a comment that mentions the agent; pushing a fix commitGitHub only
B. A headless commandA step that runs claude -p with flagsAny CI system; an agent with one fixed job, such as triageYou install the tool, supply the credentials and post the result yourself
C. A program built on the Agent SDKA step that runs your own Python or TypeScript programAn agent with custom logic around the model: several stages, your own retries, your own tool codeThe most code to own
D. A managed serviceNothing. An administrator turns it on for the repository.Managed code review; cloud sessions that fix a pull requestFewer choices. Managed review, for example, picks its own models.

Most teams start with A for review, because it needs the least work, and move a job to B when it has to run somewhere other than GitHub or needs exact control over what is loaded.

6. A run that finds its files, and a run that is told#

Section 2 said that a CI run loads the repository's files "from the checkout". That is true today, by default. It is worth knowing that this is a choice, because the default is expected to change.

A headless run can work in two ways. In the first, it discovers its configuration: it looks in the working folder, as an interactive session would, and loads the project instructions, settings, hooks, skills, subagents and tool servers it finds. The reviewer depends on this.

In the second, called bare mode, it loads nothing by itself. Each piece must be passed with a flag. That sounds like extra work, and it is. The benefit is that the run behaves the same on every machine, because nothing it happens to find can change it.

Discover (the default today)Bare (the flag --bare)
What is loadedWhatever the working folder holds, as in an interactive sessionOnly what a flag names
FitsAgents that must follow the project's rules: the reviewer, the fix-on-request agentRuns that must be identical everywhere and start fast
The riskThe run obeys whatever the checkout contains, including files a pull request changedA piece you forgot to pass is silently absent
HooksRunDo not run, even when passed with a flag
SkillsListed to the model, which picks from themLoaded from a named folder, but the model is not shown the list. Name the skill in the prompt, or put the instructions in the system prompt.
CredentialsAny supported loginAn API key or a cloud provider. No subscription login.
To pass this in bare modeUse this flag
Instructions--append-system-prompt or --append-system-prompt-file
Settings--settings
Tool servers--mcp-config
Subagent definitions--agents
A plugin folder--plugin-dir

Claude Code's documentation recommends bare mode for scripted runs and says it will become the default for headless runs in a later release. The practical lesson: do not let a CI command depend on the default. Either pass --bare and name every piece, or accept discovery knowingly. A command that works today only because the agent happens to find a skill will stop working, without an error, on the day the default changes.

7. The pull request that rewrites its own reviewer#

Discovery has a second consequence, and this one is a security problem.

The reviewer loads its rules from the checkout. On a pull request event, the checkout is the pull request's own code. Now suppose a pull request changes one more file than it claims to: it edits review-rules/SKILL.md and deletes the rule about API contracts. The reviewer starts, loads the edited rules, and reviews the pull request by them. The change has rewritten its own judge.

Nobody needs bad intent for this to happen. A coding agent that is told to "make the review pass" may find that editing the rules is the shortest path.

DefenseHow it works
Load the agent's files from the base branchCheck out the base branch into a second folder, and take the rules and the agent definition from there. The pull request's diff is then only the thing being reviewed, never the instructions.
Make those files gate filesThe gate files (.github/, CODEOWNERS, CLAUDE.md, .claude/, .mcp.json, .githooks/, agents/ and ci/) get an owner in the owners file. A change to any of them puts the pull request in the red lane and needs that owner's approval (Review). Acme's review workflow relies on this defense, because it loads its rules from the pull request's checkout.
Give the reviewer nothing to loseNo editing tools, a shell limited to two git subcommands, no secret beyond the model key, and a token that can only read code and write comments
Do not make the agent's verdict the gateThe required checks and the owner rule decide the merge. The review informs people.

The first defense removes the problem. The other three limit the damage if it is missed. Managed code review applies the first one by design: it reads its review rules from the base branch, never from the pull request.

8. What else changes when nobody is at the keyboard#

The missing home folder was the first surprise. It is one of several differences with the same root: on a laptop a person is present, and in CI nobody is.

On your machineIn CIWhat to do about it
A person answers permission questionsNobody can answerUse a permission mode that refuses what was not allowed, and an explicit list of allowed tools
You stop a run that goes wrongNothing stops itThree caps: turns, spend, and the job timeout
You read the outputA script reads itAsk for JSON, and for a fixed shape where a later step depends on it
You wrote the inputThe input is a diff, a log or a comment written by someone elseTreat it as untrusted text. It may contain a sentence that reads like an instruction.
You are signed inA secret signs the job inAn API key, a long-lived token, or a short-lived identity from a cloud provider
Commits are yoursCommits come from an automation identityA commit made with the workflow's default token does not start other workflows. Use an app identity when the checks must run again.
The full history is on diskThe checkout is shallow by defaultAsk for more history when the agent must compare with the base branch

The sign-in row needs a standard, because every example on this site makes the same choice. Prefer a short-lived identity: where the model is served through a cloud provider, the job signs in with an identity that provider issues for the run, so no long-lived key exists to leak. On GitHub, the Action also supports workload identity federation: it exchanges the workflow's short-lived GitHub token for model access, with no stored key. An API key in the CI secret store, passed only to the job that needs it, is the simpler start, and it is what the examples here use.

One permission is easy to remove by mistake. Every job that uses anthropics/claude-code-action keeps id-token: write, because the Action's documentation requires it for its default GitHub App sign-in, the identity whose commits start other workflows. A job that runs the command-line tool itself, such as the triage job in section 4, does not get it.

9. When one of these files changes#

Every file on this page can change, and so can the model behind the agent. A change to an agent is harder to judge than a change to code, because its effect cannot be read from the diff. A sentence added to a skill looks harmless and can make the agent worse at something unrelated.

Each kind of change has a different first test. The table gives the short form. Four changes are worked from start to finish, with the original, the change and the results, on When the change is an agent.

What changesExampleWhere the change is madeThe first test to run
A prompt or a skill fileA new rule in read-build-logA file in the repositoryTests of the one sub-task, then a small set of cases picked from the edit
What a tool returnsThe log tool returns 100 lines, not 200The tool server's codeA script with no model: does each test case still receive the evidence it needs?
A new toolA tool that lists dependency changesThe tool server, and the allowed-tools list in the workflowIs the tool used when it should be, and left alone otherwise?
The modelThe provider's next versionOne line in the run scriptThe full set of cases, old against new. Nothing cheaper sees it.
The workflowA wider permission, another event.github/workflows/A person. This is a gate file.

A merged change does not take effect by itself. .github/agent-versions.yml, a gate file owned by @acme/platform-leads, names the active version of each pipeline agent; a new version runs in shadow until it is switched there (the lifecycle). On a pull request, the first tests run as the agent checks skill-lint, trigger-test and agent-smoke; the full golden set runs as agent-eval in the merge queue.

The tools in these examples come from tool servers (section 2). A tool server can wrap anything: the CI system's logs, the list of changes, a bug tracker. A host's own tool server is one option; a company's internal systems usually get their own.

10. Four questions for any pipeline agent#

The reviewer and the triage agent look different, and they were set up by answering the same four questions. Which event starts it? Which files define it? Which command starts it? What may it write?

The same four questions describe every agent a team is likely to put in a pipeline.

#AgentStarts onDefined inStarted byMay write
1ReviewerA pull request opens or changes.claude/agents/code-reviewer.md and a skillThe Action, with the agent and its allowed tools named in claude_argsComments only
2Fix-on-request agentA comment that mentions the agent.claude/agents/fix-finding.mdThe Action, in its mode that answers mentionsCommits to that branch
3Triage of a red buildThe test workflow fails on mainagents/triage/A headless commandNothing. It writes a claim file.
4Red-build fix agentThe verifier reports "supported"agents/fix/A headless command; a script then opens the pull requestIts own branch
5Security reviewA pull request touches a sensitive pathA command bundled with the agentThe Action, with that command as the promptComments only
6Documentation follows the codeA merge to main that changed a public APIA project skillThe Action, on a push to mainA pull request
7Dependency upgradeA scheduleA project skillThe Action, on a scheduleA pull request
8Issue sorterA new issueA project skillThe Action, on an issue eventLabels only
9Flaky-test recorderA test passed on its second runA script first; an agent only to write the recordA headless commandA bug record
10Eval runner for agent changesA pull request that touches agents/, then the merge queueagents/triage/evals/A script that runs the agent on frozen dataCheck results: agent-smoke on the pull request, agent-eval (the full golden set) in the queue

The last column is the one to read twice. For each agent, compare what it may write with what its job needs. Where the first is wider than the second, narrow it. An agent that only comments cannot break the build, however wrong it is.

11. Side questions#

QuestionShort answer
Can one skill serve both a developer and CI?Yes, and it should. The review rules a developer runs before pushing are the same file the CI reviewer loads. One file cannot drift from itself.
How do I test a change to one of these files?As a pull request with an eval. See When the change is an agent.
Is this different on GitLab?The agent's files are the same. The pipeline file is .gitlab-ci.yml, and the job runs the headless command.
What about other coding agents?Each has its own folder names for instructions and skills. The pattern holds: files in the repository, a pipeline file that starts the run, and nothing from a home folder.
Where do secrets go?In the CI system's secret store, passed to the one job that needs them. Never in a skill, a prompt or a settings file. Better still, no stored secret: a short-lived identity (section 8).

The developer in the example thought she had a CI problem. She had a level problem: a team rule stored at the personal level. Her agent was never different in the pipeline; it was only uninformed. Once the rules, the powers and the start command were in the repository, and the rules that may never vary were in managed settings, the agent became something the team could read, review and protect like any other code. That is the useful way to think about an agent in CI: it is not a service you configure somewhere else. It is a set of files at the right level, and a workflow that runs them when nobody is watching.

Next: build the agent that pinpoints a failing change, and the agent that fixes it.