> Canonical: https://antifailure.dev > A disposable copy of your production stack for every pull request: masked Postgres, contained third-party APIs, and agents that use your app like people. # Know what happens before you deploy, on a disposable production twin. The engine is open source and runs in your own continuous integration today, with no account. The hosted control plane is a managed service: book a demo and we set your team up on it. - Isolated Twin . A temporary copy of the application stack for every risky change. baseline idle prod.internal ttl 12:41 candidate live fix-billing-184 ttl 12:38 candidate live fix-billing-184 ttl 12:38 api worker postgres isolated - Safe State . Sanitized, referentially consistent, production-shaped Postgres. public.users unique u_8f2a ajay@acme.com u_91c0 s***@example.net u_bb12 l***@example.net refs u_8f2a m***@example.net orders.user_id sessions.uid - Side-Effect Firewall . A stateful Stripe and captured mail instead of charging cards. charge email CHG-184 stripe.charges MAIL-91 sendgrid.send DNS-018 api.prod.internal egress DNS-018 api.prod.internal denied unknown dest - Load . Traffic shaped like production's own access log, sent at the twin. mix not live diversion GET /settings/billing 34% GET / 27% GET /api/subscriptions 129% slower 18% not sent no rule named them safe 1 POST /billing/upgrade 2 POST /api/payments 3 DELETE /api/seats - Migration Safety . Locks, table rewrites, and query plans before it ships. migration When exclusive lock 0.0 s Then watching plan LOCK QUEUE REWRITE PLAN FINDING 27.4s · blocked a session ## A risky pull request, stopped before it merges. Eighty five seconds. Antifailure makes a copy of production, the same size and the same shape and the same load, with every real name replaced. Your change runs there whether a person wrote it or an agent did. - A copy of production, same size and shape - Your change runs there first - Then the copy deletes itself - Migration Safety - Isolated Twin - Safety properties - Load - Side-Effect Firewall ## Catch exclusive locks before they take checkout down. The strongest lock held per table, whether another session was left waiting on it, and how the plans moved. - Pulse - Inbox - My issues - Reviews Workspace - Initiatives - Projects - More Favorites - widen_plan_id - checkout replay - table rewrite - lint rules ## widen_plan_id Widen plan_id on subscriptions and measure the exclusive lock, whether another session is left waiting on it, and whether the table is rewritten before the change ships. Activity - Antifailure opened the run from checkout traffic · 2min ago - The run added the labels FINDING and Locks · 2min ago - maya · 4min ago ACCESS EXCLUSIVE held 27.4s, and another session was seen waiting on it even on the branch. - jordan · just now @antifailure can you take a stab at this? - Antifailure connected · just now - Changed 2 files · nullable add, backfill later · just now - Antifailure moved from Todo to Block · just now subscriptions schema in this run Ran for 10 s · data stayed in-boundary - Locks FOUND ACCESS EXCLUSIVE on subscriptions, held for 27.4s, and another session waited on it. - Plans FOUND Checkout reads fell back to Seq Scan. Planner dropped subscriptions_status_idx. - Rewrite FOUND int to bigint rewrites every row of subscriptions. Postgres reports it, we do not guess. - Lint FOUND Changing a column to bigint rewrites the whole table under ACCESS EXCLUSIVE. The rule carries the fix. - Cleanup OK Branch torn down. Production data never left the customer boundary. Verdict FINDING Do not ship widen_plan_id as written Safer path: a second column of the new type, no table rewrite, backfill in small batches Example finding One migration rehearsed, with the numbers chosen. The hold time and whether the table was rewritten are measured too. ## A disposable production twin. Build the candidate, restore safe state, contain side effects, and destroy everything when the report is done. Isolated twin Production not in path - Build - Restore - Contain - Destroy ### Production - ingress - app - prod-db - workers ### Containment The twin cannot reach live keys or the public internet. - Clone-local DNS No public resolver - Secrets replaced Live keys unreachable - No default egress The network has no route out ### Disposable twin - Network waiting - App waiting - Postgres waiting - Workers waiting Resources are journaled as they come up. Nothing outlives the run. Illustrative The order is real, and so are the containment seals: build, restore safe state, contain, destroy. The percentages are a shaped run. - Isolated networking Clone-local DNS, no default public egress, no route out of the network. - Safe credentials Production secrets are replaced. The twin cannot reach live keys. - Cleanup proof Every resource is journaled, destroyed, and counted at teardown. Nothing outlives the run. ## Whether this deployment is safe to ship against real data, real concurrency, real workers, and the deploy itself. env-4c1 reached for an unlisted host No rule for it. Denied, and written to the log. ### Fail closed. Unknown destinations are denied inside the twin, and an unverified golden cannot be branched. ### Traffic shaped like production's. The route mix out of your own access log, with the worst regression first. ### Pass or fail, with evidence. A gate on the pull request carrying the rows and the trace behind it. ## Traffic shaped like production's. The route mix out of your own access log, sent at the twin. Not production traffic diverted. Three commands. af init reads the repository and writes the manifest, af up builds the twin around a branch of the golden, af test runs the workflows and returns verdicts with evidence. With a production database, af golden refresh once before af up. - Read the repository, wrote antifailure.yaml - Branched a verified golden for this change - Contained Stripe and email inside the twin - Ran the declared workflows, attached the report Try for yourself, start proving a change before it ships. Illustrative An editor with the real files in it. af init, af up and af ci exist and do what the terminal shows; the repository and its contents are made up. ## Fail closed on side effects. The twin cannot charge cards, email users, or invoke production webhooks. Unknown destinations are blocked. - POST /v1/charges $49.00 CHG-184 Stripe simulated - Retry charge on checkout CHG-185 Stripe simulated - Order #4182 receipt MAIL-91 Email captured - slack.hooks · store only WH-220 Webhook captured - api.prod.internal DNS-018 Production blocked - 18.4.2.9 · ip-bypass TCP-443 Unknown blocked - hooks.prod.internal WH-441 Webhook blocked Twin posted POST /v1/charges $49.00. Simulated against clone-local Stripe. Not live. Yes, SendGrid /v3/mail/send is captured. MIME stored. Never delivered. hooks.prod.internal is unresolved. Unknown destination, denied. Fail closed. ## Fail closed. Customer-hosted. Cleanup is journaled as it happens, not reconstructed afterwards. - Fail closed An unverified golden cannot be branched, and inside the twin there is no route out. - Customer-hosted Production data stays inside the customer boundary. The control plane never needs a copy. > A verdict is only worth what it carries. Every result on a pull request arrives with the rows it read, the trace it took and the recording of the attempt, so a reviewer can check the finding instead of trusting it. Measurable evidence, or no claim. Product brief, section 18 ## Know what happens before you deploy. Create a disposable production twin for every risky change. Catch migration failures before they reach customers.