# Running a workload from the console

URL: https://antifailure.dev/docs/guides/load-console

The Load area of the console, the four kinds of workload, and what every number on a result means.

---

`af load run` prints a summary and exits. That is the right amount of output
for one run on your own machine. It is the wrong amount when you want to
compare this week against last week, hand a colleague the evidence, or answer
"was that route always this slow".

The Load area of the console keeps every run, and shows what each one measured
rather than a summary of it.

## What a workload is

A workload is something the engine can run against a disposable twin. There are
four kinds, and the console keeps them apart because they measure materially
different things: a mix has no order, a journey has no browser, a workflow has
no request rate, and an exploration has no pass.

| Kind | Command | What it is | How exactly it replays |
| --- | --- | --- | --- |
| Observed load | `af load run` | A weighted mix compiled from an OTLP export or an access log, so it is the endpoint mix production actually served | As a shape. The mix and the rate replay and the picker is seeded, so two runs send the same sequence. The individual production requests do not replay. |
| Scenario | `af load scenario` | Journeys written into your manifest and selected by name: steps in an order, with think time between them | Exactly. The same scenarios at the same seed plan the same requests in the same order. |
| Workflow | `af test` | Workflows out of your manifest, driven in a real browser by an agent | As an outcome rather than as a sequence. An agent reads the page it is on, so two runs can reach the same result by different routes. |
| Exploration | `af explore` | An agent choosing its own way through the application from a goal and a seed | At the same seed, the same wander. |

The console states the reproducibility of a kind above its numbers. A scenario
that replays request for request and a mix that replays only as a shape are not
equally strong evidence, and that difference matters more when somebody
disagrees with the result than when they agree with it.

The first two are described in full under [Load](/docs/concepts/load), the
third under [Workflows](/docs/guides/workflows) and the fourth under
[Exploration](/docs/concepts/exploration).

### A workload names things; it does not contain them

This is the part that surprises people. A workload does not carry a scenario
document or a journey. Every runnable thing is declared in **your** manifest
and selected by name, the same way the command line selects one:

```
af load scenario --only checkout
af test --only sign-up
af explore --only upgrade-a-plan
```

So a workload is a selection plus the knobs its command actually declares. That
is smaller than it first appears and it is the whole of what is real. It is
also a security property rather than a simplification: a scenario is checked
against your manifest's safe route list before anything is sent, and a control
plane able to hand an engine an arbitrary journey would be a control plane that
can send traffic you never allowed.

## Versions

Changing what a workload runs writes a new **version** beside the old one.
Versions are immutable, every run records which one it used, and that is what
makes a run from three weeks ago readable at all.

It also makes comparison work. Running a mix at scale 1 and at scale 4 is two
versions of one workload, so comparing those runs is comparing two versions
rather than two runs whose settings live only in a form somebody has closed.

Saving a form you did not change adds nothing, and the console says so rather
than filling the history with entries that differ in nothing.

### Which knobs each kind has

A knob exists only when the kind's command has a flag for it. Offering one it
does not have would be a control that exists to be refused, so the console does
not draw it and says why underneath.

| Knob | Observed load | Scenario | Workflow | Exploration |
| --- | --- | --- | --- | --- |
| Selection (`--only`) | not applicable | required | optional | required |
| Duration | yes | no | no | no |
| Scale | yes | no | no | no |
| Concurrency | no | yes | no | no |
| Seed | no | a whole number | no | any string |

**Scale** multiplies production's rate, so an observed mix at scale 1 arrives
at the rate production served it, and **duration** bounds the run. Neither
exists for a scenario, which runs its steps in order for as long as they take
rather than sending at a rate.

**Concurrency** caps requests in flight for a scenario. `af load run` has no
such flag, so an observed mix cannot set it: accepting the knob and running at
the generator's own default would produce a run that did not do what its author
asked, with nothing in the result saying so.

**Seed** makes two runs send the same schedule or walk the same way, which is
what makes one run comparable with another. `af load run` takes one on the
command line and a version cannot: a dispatch carries the inputs the workflow
file declares, and the four-input workflow this product shipped before the
console could start a run has no seed among them. Sending an input a workflow
does not declare is a 422 from GitHub.

**A selection is required for a scenario and for an exploration**, and an empty
one is refused. Their commands default to everything the manifest declares, so
a manifest that later gains a scenario would silently change what a saved
workload runs. `af test` genuinely means every workflow when it is given no
`--only`, so a workflow workload may leave it empty and the console says what
that means.

## Starting a run

Open a workload and use **Start a run**. It takes an environment and a version,
and nothing else, because every knob is in the version.

The environment has to belong to the same repository as the workload. A
workload names routes and workflows out of one repository's manifest, so
running it against another one measures nothing, and the console offers only
the environments that can work.

**Nothing runs in the control plane.** Starting a run asks GitHub to run
`.github/workflows/antifailure.yml` in your own repository, on the
environment's own branch. That is what keeps your database, your secrets and
your third-party credentials inside your own cloud. See
[GitHub](/docs/guides/github) for the workflow file itself.

Two of the four kinds need inputs that the workflow gained when the console
learned to start runs. Against an older copy GitHub refuses the dispatch, and
because it reads the trigger definition from your repository's **default
branch**, adding the newer file on a feature branch alone does nothing. The run
is recorded either way, carrying the refusal, so a dispatch that never happened
is visible rather than silent.

### Safe and unsafe routes are a manifest decision

They are not on this form, and that is deliberate. No load command has a
`--safe` or `--unsafe` flag: the lists live under `load` in your manifest, so
they are committed alongside the code and reviewed with it rather than being
set per run.

The rule they express is the one worth reading twice. Every route is unsafe
until a safe pattern matches it, because a generator that finds
`POST /checkout` in an access log and runs it four hundred times charges four
hundred cards. A pattern is a method and a path glob, as in `GET /api/*`, where
`*` covers one segment and `**` covers the rest; a bare glob matches any
method. An empty safe list sends nothing at all.

A run's result lists the routes the safe list refused. A run that sent less
than you expected usually means the safe list is too narrow rather than that
the traffic was not there, and that list is how you tell.

## Where a run is, and what it found

A run carries a **state** and a **verdict**, and they answer different
questions. Neither implies the other: a run can do all its work cleanly and
fail every threshold in it, which is `succeeded` and `fail`.

| State | Means |
| --- | --- |
| `requested` | Recorded here and asked of GitHub Actions. No engine has picked it up yet, so nothing is running. |
| `accepted` | An engine has claimed the run and is bringing the environment up. |
| `running` | The engine is doing the work now. |
| `succeeded` | The engine did the work and reported. What it found is the verdict. |
| `failed` | The engine reported that the work itself failed. |
| `cancelled` | Stopped before it finished. |
| `timed_out` | The engine reported that it ran out of time. |
| `abandoned` | The deadline passed with no engine reporting. |

**`abandoned` is not a failure.** A failure is something an engine told us;
this is the control plane admitting it never heard. The run may well have
happened, and what is missing is the report rather than necessarily the work.
The two want different things done about them: a failed run is a defect in the
change, and an abandoned one is a defect in the plumbing.

### The commonest reason a run never reports

`af` does the work with or without a token. An environment comes up, the agents
run, the report is written. What the token decides is whether any of that is
**reported** back here.

Without `AF_CONTROL_PLANE_TOKEN` the engine claims no hosted run and sends no
events, so a run you start from the console is dispatched, actually runs, does
everything you asked, and ends `abandoned` at its deadline. Nothing is wrong
with your software and nothing is wrong with the run. The console simply never
heard about it.

Make one with `af token create ci` and add it to the repository's secrets under
that name. The workflow reads it as `secrets.AF_CONTROL_PLANE_TOKEN`. Leave it
out and everything except the hosted reporting keeps working, which is the
self-hosted path and stays supported.

The console says this on the run itself, and only where it applies: on a run
nothing ever claimed. A run an engine **did** claim and then went quiet on is a
different problem, and the console says which two things it cannot tell apart
rather than guessing between them. An engine that loses its lease to a second
engine stops and deliberately says nothing, rather than ending a run the second
one is now doing and destroying its measurements, so a lost lease and a dead
runner look identical from here. The Actions run for the branch is where to
look next.

### A run waiting to be claimed

`requested` with a dispatch behind it is neither running nor an error, and the
console says so rather than leaving it looking like a hang. A GitHub Actions
job has to start, check the code out and reach the control plane, so a minute
or two is ordinary. Much longer than that is usually the token above.

| Verdict | Means |
| --- | --- |
| `pass` | Everything that was evaluated held. |
| `fail` | At least one thing was evaluated and did not hold, and it stops a merge. |
| `flaky` | The same check answered differently on repeat. |
| `warn` | A real finding that does not stop a merge. |
| `blocked` | The work never reached the application, so nothing measured is a judgement about it. |
| `unverified` | It finished and nothing could be evaluated, so it proved nothing either way. |

**Only `pass` is a pass**, and the console never draws any of the other five as
one. If you are gating anything on a result, gate on `pass` rather than on the
absence of `fail`.

Four of them are drawn in the same amber, and two pairs of those mean opposite
things, so the console puts a sentence under the badge rather than leaving the
colour to carry it. `flaky` and `warn` mean something was looked at and
something was found. `blocked` and `unverified` mean nothing was looked at. The
first pair is a finding about your change; the second is a gap in the run.

When a recorded verdict disagrees with the thresholds under it, a pass over
something that broke, or came back flaky, or was never evaluated, the console
says so above the table. It cannot correct a verdict the engine computed, but
it will not show you the contradiction quietly.

A failing run always says what failed it, beside the verdict. That is not
decoration: a load run that sent traffic and broke a threshold carries no
message of its own, because the engine writes one only when nothing was sent.
Left alone it would be a red word with nothing next to it. So the console names
the thresholds that broke, out of the rows that recorded them, and when none of
them did it says that instead rather than going quiet.

## Reading the result

Nothing is written until a run reaches an end, so a run that is still going has
no result at all rather than a partly filled one. The console says which.

### Did it keep up

For a run that sent traffic, the first number is the achieved rate against the
rate that was asked for. A run that asked for 200 requests a second and
achieved 60 has already found something, before any latency figure is read,
because every latency figure under it was then measured behind a queue. The
console says so outright when a run falls more than a tenth short.

### Did anything get checked

For a browser workflow the console shows five counts, not two: passed, failed,
flaky, blocked and unverified. A run with workflows to drive and none passed,
none failed and none flaky checked nothing at all, which is not the same as
nothing being wrong, and the console says that outright. With passed and failed
alone it would have drawn as a run with no failures.

### Latency

Five percentiles: p50, p90, p95, p99 and max. Percentiles rather than an
average, because an average hides the tail and the tail is what a user notices.
A p50 that halves while the p99 doubles is a regression an average reports as
an improvement.

A percentile the run did not record is absent from the ladder. It is never
drawn at zero, because a p99 of nothing and an unmeasured p99 are different
facts.

### Errors, by reason

The error count is broken out by reason rather than totalled. A thousand
timeouts and a thousand refused connections are the same number and completely
different problems.

| Reason | What it usually means |
| --- | --- |
| `timeout` | The application did not answer inside the request deadline. |
| `connection refused` | Nothing was listening. Usually the service is not up yet. |
| `connection reset` | The connection was closed mid-request, often a crash or a restart. |
| `name not resolved` | DNS did not answer for the host, which under a deny-all egress policy is what a blocked host looks like. |
| `malformed request` | The request could not be built. This is the scenario or the mix, not the application. |
| `request failed` | A transport error the runner could not classify further. |

An HTTP status of 500 or above arrives spelled as its number.

### Routes against production

Each route is compared against production's own p95, worst regression first, so
the answer is the first row rather than something to read the whole table for.

A route with no production baseline says **no baseline**. It does not say "no
change", and it can never count as a regression. Comparing against nothing and
calling the answer a regression is how a check becomes noise.

A run that selected more than one scenario carries the scenario beside the
route, because two scenarios can send the same route and their two p95 values
do not average into a p95.

### Thresholds

Each threshold carries the same five verdicts a run does, and shows the limit
it declared beside what was measured against it. The limit and the observation
are blank for `every_request_succeeded` and `status_in`, which are not numeric
comparisons; the observation alone is blank when nothing was sent, which is a
different answer from an observation of zero.

### Evidence

What the run left behind, and whether it can still be read. Three answers, not
two:

| Availability | Means |
| --- | --- |
| Kept | Stored, with a checksum to verify it against. |
| On the runner | Written to a path on the CI runner and never uploaded. The machine is gone, so the path is a record of where it was rather than somewhere to fetch it from. |
| Dropped | It existed and retention did not keep it. |

A path on a runner is never drawn as a link. Reports in this product have
carried exactly those paths, and a link to one sends you to a 404 and blames
itself.

## Stopping and repeating a run

The control plane cannot reach a runtime, so **Stop this run** is a durable
command with a deadline rather than a flag. A run nothing has claimed yet is
over immediately; anything else waits for a runtime to confirm, and the console
shows where that request got to. If the deadline passes with nothing
acknowledging it, it says the stop was never confirmed rather than showing you
a cancelled run that may still be going out there.

A stopped run keeps whatever it measured, labelled as covering only the part
that ran. Those numbers are real and they are not a measurement of the whole
run.

**Run it again** runs the **same version**, deliberately, and not the latest. A
retry answers "was that a fluke", and answering it with a definition somebody
edited in the meantime answers a different question while looking like it
answered this one. Running the latest is Start, which is a different button. A
run can be retried once: two independent successors to one failure is a history
nobody can read, and the console links to the one that already exists.

Every run an engine reported on shows the command that reproduces it, exactly
as the engine reported it. It is not rebuilt from the version, so it cannot
drift from the one that actually ran, and a run nothing reported shows no
command rather than a plausible one.

## Promoting an exploration

An exploration finds a route nobody wrote down. Promotion compiles what it
found into a **workflow** for your manifest, which `af test` runs. It never
produces a load scenario: nothing in an exploration record carries a rate.

Paste the document `af explore --output json` printed. It lives on
whichever machine ran the command and nothing sends it here on its own.

That document is an envelope with one entry per goal, so a run that explored
two goals gives you two explorations and the console asks which one to compile.
It says what each is worth before you choose: an exploration that never reached
its goal still compiles, and the workflow it produces asserts something nobody
has seen happen, so it comes back `unverified` until the path exists. A
`blocked` one is called out more sharply, because nothing was explored at all
and `af explore --emit-workflow` skips those outright.

A single exploration lifted out of the array works too, if that is what you
have.

Two things come back with the new version and neither is decoration.

**What the compilation did not carry.** This list is never empty. It always
carries at least the note that the expectation is the goal, because an
exploration knows what it was looking for and does not know what a passing page
should say; a workflow whose expectation cannot be read comes back `unverified`
rather than as a pass. It also names every friction finding it refused to turn
into an expectation, because "pressing Upgrade plan changes nothing" is a
defect to fix rather than an outcome to assert, and it says how much of the
application was left unexplored.

**The block to paste into `antifailure.yaml`.** Until that block is committed,
`af test` cannot find the workflow the new version selects, even when you
name it with `--only`, and the run comes back saying so. The control plane
cannot put a file in your repository, and it says that rather than returning
a name and letting you find out.

Copy copies the block and nothing else. The notes above it are deliberately not
emitted as YAML comments inside it, because a comment pasted into a manifest
stays there forever; they belong beside the block, once, on the screen where
somebody decides whether to keep the promotion.

## Who can do what

| Permission | Held by | Lets you |
| --- | --- | --- |
| `workloads.view` | every role | read workloads, their versions and their runs |
| `workloads.edit` | owner, admin, member | create a workload, add a version, archive one, promote an exploration |
| `workloads.run` | owner, admin, member | start, stop and repeat a run |

A viewer sees the runs and their results, and is told which control their role
cannot use rather than being shown a page with the control missing. A control
that always answers "your role cannot do this" is worse than no control, and a
missing one leaves somebody unable to tell whether the product lacks the
feature or their role lacks the permission.

Archiving hides a workload from the list and deletes nothing: every run of it
stays readable, and its versions are what those runs mean. A workload with a
run still going cannot be archived, because that would hide the run somebody
may need to stop.