An environment per pull request

The quickstart gets an environment running on your machine. This gets one running on every pull request, reported back on the pull request itself. Each push builds your services, branches a masked copy of your production database, runs the agents through your workflows, rehearses the migrations, and leaves one comment that it edits in place. It needs a repository on GitHub and nothing else: no account, no control plane, no server to host, and no secret to create before the first check runs.

Three ways in, pick one

Install the GitHub App. When the App is installed on a repository that has no workflow, it opens a pull request titled “Check every pull request with Antifailure” on a branch called antifailure/setup. The pull request adds one file. Merge it, and the next pull request gets a check. The console lists the repositories it is still getting connected. The pull request the App opens says what happens when the App cannot write to the repository.

Run af init. When the checkout has a github.com remote, af init writes the same file to .github/workflows/antifailure.yml and lists it under “Written”, beside the manifest. It also adds the github block to the draft. A project that already has a manifest gets the file from af github init, which is idempotent and refuses to replace a file that differs unless you pass --force. Both print the secrets that are optional and the one variable the hosted control plane needs.

Copy it by hand. The file is examples/github-workflow.yml in the repository. Copy it to .github/workflows/antifailure.yml and commit.

The file

This is the whole of what lands in your repository:

# Antifailure checks every pull request on a disposable copy of production.
#
# `af init` writes this file for you, and so does installing the GitHub App.
# Copying it to .github/workflows/antifailure.yml by hand works too. The work
# happens in the reusable workflow it calls, so this file rarely needs to change.
#
# https://antifailure.dev/docs/getting-started/pull-requests
name: Antifailure

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]
  # Only the hosted control plane uses this. Its buttons run this workflow on
  # the branch an environment is on. Delete it if you do not use one.
  workflow_dispatch:
    inputs:
      command: { type: choice, default: up, options: [up, down, agents, load, scenario, explore], description: "Which part to run" }
      workflows: { description: "Comma separated names out of the manifest. Empty means all of them." }
      duration: { description: "How long to send load for, as a Go duration such as 60s" }
      scale: { description: "Multiplier on production's rate" }
      seed: { description: "Makes two runs do the same thing" }
      concurrency: { description: "Ceiling on requests in flight" }
      run_id: { description: "Leave it empty. The engine asks." }

permissions:
  contents: read
  pull-requests: write
  id-token: write

jobs:
  check:
    uses: antifailure/antifailure/.github/workflows/check.yml@v1
    secrets: inherit
    with:
      dispatch: ${{ toJSON(inputs) }}
      # Where the run reports: the control plane whose GitHub App posts the
      # check on this pull request, so the check is answered by this run rather
      # than by a timeout. Set the variable to point the run somewhere else.
      control-plane: ${{ vars.AF_CONTROL_PLANE || 'https://app.antifailure.dev' }}

The App writes its own address on that last line. The file above carries the hosted control plane’s, and a self hosted control plane that knows its public address writes that instead.

The job calls a reusable workflow in the Antifailure repository. That workflow checks out your branch with full history, because af change diffs against the merge base. It applies the fork label gate, sets the concurrency group so a push cancels the check it supersedes, and then calls the action.

The action, antifailure/antifailure@v1, installs af, installs the agent runner when the command needs a browser, works out what the change touches, runs the check, and leaves the comment. Its inputs and outputs are on the action reference.

secrets: inherit lets the reusable workflow see your secrets, and it reads only the ones the manifest names. af change reports which those are before the check starts, and each is looked up by that name and passed to the action under it. A secret the manifest never mentions is never read.

The permissions block is what the job needs: pull-requests: write for the comment, and id-token: write so the job can prove who it is to a control plane without a stored credential.

Nothing else is required

No secrets and no account. Open a pull request and the workflow runs, af change reads the diff, and af ci brings the environment up, runs the workflows, asks the invariants, rehearses the migrations, writes the report and tears down. Teardown happens whatever the outcome, including on a failed job and on a cancelled one.

af change is what keeps the check off a change to a README. It reads the diff, says which checks exercise what it touched, and writes that as the comment when nothing else runs. A path it does not recognise selects every check rather than none.

What is optional, by name

Each of these is a repository secret, except the last, which is a repository variable. Each is read only when the manifest asks for it.

ANTHROPIC_API_KEY lets the agents read a page. Without one they still run, and a workflow that needed a page read comes back unverified rather than guessed at. On a workstation, af model set anthropic keeps the key out of your shell profile; see your own model key.

AF_MASKING_KEY makes masking deterministic across machines, so two goldens can be compared. Left unset, every runner generates its own.

The production database secret has whatever name the manifest’s database.source_url_env chooses, such as PRODUCTION_DATABASE_URL. Add a secret of that name and the workflow passes it automatically, because the action reads the manifest and exports the variable it names. Nothing in the workflow file changes when the name does. Without it the check runs on an empty database, and the report says so at the top.

STRIPE_TEST_SECRET_KEY is needed only when the manifest sets a host to sandbox mode. The action exports it as STRIPE_SECRET_KEY, which is the name the engine reads. It has to be a test key. A live one is refused before anything starts.

AF_CONTROL_PLANE is a repository variable, not a secret. The file already carries one as the variable’s default: the control plane whose App opened the pull request, or the hosted one when you copied the file by hand. The run reports there, and the control plane concludes the check it posted and maintains the comment, so there is nothing to set. Set the variable only to point the run at a self hosted control plane. A repository the control plane does not know refuses the run a credential, the job comments for itself, and nothing is red for it. The control plane is what reporting adds.

No manifest yet

The check does not wait for one. When the repository has no antifailure.yaml, af ci drafts a manifest from the repository, in memory, the same way af init would, and uses that. The comment says so in its first lines: this run used a manifest Antifailure drafted from the repository, and af init committed is what makes it yours. A repository the draft cannot describe gets a skipped run and a comment naming the reason, with af init as the next command.

An empty database

When database.source_url_env is unset, the report opens with this sentence:

This ran on an empty database. database.source_url_env names nothing, so the migrations built the schema and no production data was masked or branched. Set database.source_url_env: PRODUCTION_DATABASE_URL and add that secret to the repository.

It is rendered before the workflow table. af up prints the same sentence on a workstation.

Turn the integration on

af init adds this to the manifest when it writes the workflow. Add it by hand if you copied the file:

github:
  mode: actions
  comment: true
  fork_policy: label

There is a teardown_on key as well, and it is read by nothing: teardown happens whatever you put there.

mode: actions runs everything inside the workflow, and the environment lives for the length of the job. For preview URLs somebody opens later, the control plane is what adds them, and the mode becomes app.

Open a pull request

Push the branch and open one. The workflow runs and leaves a single comment. It carries a headline saying what the run amounted to, the environment URL, and a row per workflow with its verdict and the detail behind it. Below that sit a collapsible set of steps for reproducing any workflow that did not pass, and a footer naming the branch, the commit, how long it took and which golden it branched from.

It also carries what the data said: every invariant the manifest declares is asked after the workflows, and a violated one puts the offending rows in the comment.

And it carries what this change does to the database. The pending migrations are rehearsed against a throwaway branch of the golden, and the comment names what they locked and for how long, what Postgres rewrote, and what the lint objected to. A lock held past two seconds fails the check by default; a rewrite warns. The policy block is where you change that.

It edits that comment in place on the next push rather than adding another.

Pull requests from forks

fork_policy: label is the default. Nothing runs on a pull request from a fork until a maintainer adds the antifailure:allow label. The file subscribes to labeled and unlabeled so that the approval, and a withdrawn approval, reach the check without waiting for the next push.

The policy is read from the base branch rather than from the pull request, because the pull request’s copy of the manifest belongs to the contributor. Forks has the full picture.