Change analysis

Every check in this product costs something: a branch of a golden, a build, a browser, a few minutes of a runner. Running all of it on a change to a README is waste, and running the default on a change that adds a column and edits the billing service is not enough attention.

af change reads the diff and says which checks will exercise what it touched. It names the file and the rule behind every line of it, so the reasoning can be argued with.

af change                          against the base branch this job names
af change --base origin/main       against a ref you choose
af change --diff pr.patch          against a diff you already have

4 files changed, touching the schema, the api service and an outbound host. 5 checks will run, and 1 more is selected and not configured.

  run   environment  api/billing.ts: the manifest declares the service api at the repository root, so every file in the repository is part of it (and 7 more)
  run   migration    migrations/20260824_add_billing_status.sql: the path is inside a migrations directory
  run   invariants   migrations/20260824_add_billing_status.sql: the path is inside a migrations directory
  run   workflows    api/billing.ts: the manifest declares the service api at the repository root, so every file in the repository is part of it (and 6 more)
  gap   load         load is off in the manifest, so af ci runs it only when it is handed --load
  run   egress       api/billing.ts: an added line names api.stripe.com, which the manifest routes to mode mock (and 1 more)
  skip  masking      nothing this change touches is exercised by it

Each line names one reason and counts the rest, because a check selected by eight files does not need eight sentences to justify it. af change -o json carries all of them.

What it will not tell you

It does not say whether a change is safe, and it does not grade it. There is no score and no risk word in the output, because both would be a judgement made from a file listing, and this product’s whole argument is that judgement comes from running the thing.

What it produces is one shape of sentence: this file is X, and X is exercised by check Y. Every conclusion carries the path that produced it and the name of the rule that fired, so a wrong classification can be found and corrected rather than argued with.

A path nothing recognises runs everything

If any changed path matches no rule, every check is selected. The same is true of a diff too large to classify and of a diff with no files in it, which is either an empty change or the wrong base ref, and nothing here can tell those apart.

This is a deliberate asymmetry. A path wrongly classified as documentation skips work that should have happened and nobody finds out; a path wrongly treated as unknown costs a run that was not needed and is visible in the report. Only one of those two mistakes is discoverable.

The consequence to expect: a repository with an unusual layout will select everything until its manifest says otherwise, and that is the intended behaviour rather than a bug to file.

The checks and what selects them

Surface	What it is	Selects
schema	a migration directory, a .sql file, a schema a migration tool reads	environment, migration, invariants, load
service	a file under a path a service in the manifest declares	environment, workflows
code	application source	environment, workflows, load
auth	who may do what: a guard, middleware, a session, a policy, an entitlement	environment, workflows
asset	something the application serves: a stylesheet, an image, a template	environment, workflows
build	a Dockerfile, a compose file, a build configuration	environment
dependency	a package manifest or a lockfile	environment, egress
config	configuration the application reads	environment, workflows
manifest	antifailure.yaml itself	environment, egress
masking	the masking rules file the manifest names	masking
egress	an outbound host named in an added line	egress
infrastructure	infrastructure as code	environment, workflows
database_config	a database engine version or a server parameter, in an added line of an infrastructure file	environment, migration
capacity	a replica count, an instance size or an autoscaling bound, in an added line of an infrastructure file	environment, load
network_rule	a firewall, security group or network policy rule, in an added line of an infrastructure file	environment, egress
pipeline	continuous integration configuration	nothing
test	your own test suite	nothing
docs	prose	nothing

The three surfaces that select nothing are not oversights. Prose, your own test suite and your continuous integration configuration do not run in production: nothing in a run reads a README, a workflow file or a spec, and this product runs the workflows the manifest declares rather than your test suite.

The table is not written twice. A test in the engine reads the rows above and requires them to equal the coverage table the analyser plans from, so a row here that disagrees with the engine fails the build rather than misleading a reader.

Infrastructure as code

For most of this package’s life, an infrastructure only pull request selected no check at all. Terraform sat in the same bucket as prose, your test suite and your continuous integration configuration, under one true sentence: the environment is built from antifailure.yaml rather than from your Terraform.

That sentence is still true and it was never a reason for zero. Prose, a test file and a workflow file do not run in production. Your infrastructure as code does. It was the one of the four that is not inert, and the consequence was not a quieter plan: the published action gates the whole run on the environment output, so a pull request that changed production’s database, its capacity or its firewall ran nothing.

So an infrastructure change now brings the environment up and drives the application inside it, and the added lines are read for what they actually say.

4 files changed, touching infrastructure, the database's configuration, capacity and a network rule. 4 checks will run, and 1 more is selected and not configured.

  run   environment  infra/ecs.tf: an added line sets desired_count, which is how much of the application is there to serve traffic, and load is the check that puts production shaped traffic through it (and 9 more)
  run   migration    infra/rds.tf: an added line sets a database engine version of 16 and the manifest declares 15, so the migration rehearsal applies this change's migrations to 15 and not to 16. Whether this is the database the manifest means is not visible from a diff (and 1 more)
  skip  invariants   nothing this change touches is exercised by it
  run   workflows    infra/ecs.tf: it is infrastructure as code (and 3 more)
  gap   load         load is off in the manifest, so af ci runs it only when it is handed --load
  run   egress       infra/security.tf: an added line declares the network rule aws_security_group_rule, which decides what this application may reach and what may reach it, and the egress check is where an outbound request meets the policy and gets a decision (and 2 more)
  skip  masking      nothing this change touches is exercised by it

That is the real output of af change over engine/internal/change/testdata/infrastructure.diff, against a manifest whose database.version is 15 and whose load is turned off. The gap line is load being selected by the replica count and not configured, which is the report saying that something changed and nothing is going to look at it.

Read the claim precisely, because it is narrower than it looks and the report repeats the difference on every run that touches one of these files: nothing applies your infrastructure as code. The environment stands in for the runtime the change describes, and standing in for it is not being it.

The version comparison is the sharpest line of the three. A pull request that moves production to Postgres 16 while antifailure.yaml still says 15 is rehearsed against 15, and nothing used to say so: the diff held one number, the manifest held the other, and no reader held both.

Three limits, stated rather than discovered:

• A bare version key is not read as a database version. Azure writes the Postgres major that way, so a version bump there is missed. The alternative is reading version on every provider pin, every Helm chart and every Kubernetes API line, which would select the migration rehearsal on a chart bump.
• A URL inside an infrastructure file is not read as an outbound host. In Terraform it is usually a module source or a provider registry, which is the lockfile case: a download the build makes, not a call the application makes. What a file says about the network is read instead from firewall and security group rules, which do not have to guess.
• Nothing here parses HCL. The same path rule claims Terraform, Bicep, CloudFormation, Kubernetes manifests and Helm values, and a parser for one of the five would answer nothing about the other four while reading in the report exactly like a rule that works.

Selected is not the same as available

A check is reported twice: whether this change selects it, and whether the manifest configures it at all. The interesting line is the one that is both selected and unavailable, because it means something changed and nothing is going to look at it.

gap   invariants   the manifest declares no invariants, so nothing is asked of the data after the workflows

A report that showed only “invariants: not run” would read the same whether the change did not need them or whether nobody ever wrote any.

Outbound hosts

An added line naming an http or https URL is checked against the egress policy, using the same code that decides real traffic in the sidecar. So a pull request that starts calling something new says so before the run:

egress hooks.slack.com: an added line names hooks.slack.com, which no egress
rule matches, so the default of block applies

Only added lines are read, and only in source, configuration and the manifest. A URL in a README is a link and not a call.

Teaching it your layout

The built in rules cover the conventions most projects use. A repository that puts something somewhere they do not predict declares it:

change:
  rules:
    - path: packages/*/src/**
      surface: code
    - path: ops/**
      surface: infrastructure
      note: the deployment scripts, which no environment runs

A single star does not cross a slash and a double star does. The longest matching pattern wins, so order does not decide and appending a rule cannot silently change what an existing one does.

Three things a rule cannot do. It cannot assign service, manifest, masking, egress, auth, database_config, capacity or network_rule. The first four come from declarations already in the manifest and would be a second answer to disagree with the first; the last four are conclusions drawn from reading a line rather than a path, so a rule that assigned one would be claiming to have read a file it never opened. It cannot turn a check off, because a rule says what a path is and the engine decides what that implies. And it cannot match every path: a catch all would classify everything and the fail safe above would never fire again, so the manifest refuses one.

change.rules[0].path: The change rule pattern "**" matches every path.

In a pull request check

Inside a GitHub Actions job, af change writes one output per check, so a later step can skip work this change does not need:

- id: change
  run: af change

- name: The full check
  if: steps.change.outputs.environment == 'true'
  run: af ci

The value is the check being both selected by the change and configured in the manifest, because a step asking whether to do work needs both. selected holds the same list as a comma separated string.

What a diff cannot see

Stated in the report itself, on every run, because a report that implies coverage it does not have is worse than no report:

• It reads paths and added lines. It does not run the program, so a one line change to a configuration default can change behaviour nothing here can see, and a thousand line refactor that changes nothing will still select every check its files touch.
• A caller left behind in a file the diff does not touch is invisible. The build is what finds that.
• Columns a migration adds do not exist in the golden yet, so nothing has checked whether they will need a masking rule once they carry production data. The masking check reads the golden, not the diff.
• A rename is classified by the new path, so moving a file between categories changes the classification without changing a line of code.
• A binary file has no added lines to read.
• Nothing in a run applies your infrastructure as code. The environment stands in for the runtime a Terraform change describes, so the checks it selects exercise the application in that stand in and not the change itself.
• The workflow agents drive a browser, so a change to a worker or a cron service is exercised only where the application’s own interface reaches it, and a diff cannot say whether it does.

A check that is not selected was not run. That is a statement about what was exercised, not a finding that the untouched parts are correct.

Errors

AF-DET-010 is the common one, and it is almost always a shallow checkout: a job cloned one commit deep shares no history with its base branch, so there is no merge base to diff against. fetch-depth: 0 fixes it.

AF-DET-011 means the file passed to --diff is not git’s unified format. Produce it with git diff --unified=0 base...head.