Schema changes on large tables.
The flagship wedge, felt first by teams whose users notice p99 immediately.
Measure the strongest lock held per table, how long it was held, whether another session was left waiting on it, and how the query plans moved.
Start with Postgres volume, plans, and pools, then expand.
Large tables plus frequent schema change.
Large tables. Exclusive locks and rewrites that never show up on a laptop database.
Query plans. Plan regressions under production-shaped volume.
Pools. Connection-pool exhaustion during migrate-and-serve.
- pid 1842ALTER subscriptionsACCESS EXCLUSIVEholds 27.4s
- pid 2210SELECT eventsACCESS SHAREwaiting · p99 6.9s
- poolmigrate-and-servepool waitconnections queued
27.4s hold · events p99 820ms → 6.9s
Exceptional Postgres instrumentation first.
The first supported stack should be exceptional. A broad compatibility list with unreliable connectors would destroy trust.
Postgres first. Volume, plans, and pools, then expand.
Publish what the twin reproduced. Do not pretend unsupported components are cloned.
- 01Expand
- col
- access_tier
- null
- yes
- default
- none
- 02Backfill
- batch
- 12k / pass
- dual-read
- on
- pool
- live
- 03Contract
- constraint
- last
- old path
- kept
- safe
- not yet
Publish what the twin reproduced. Do not pretend unsupported components are cloned.
Locks, plans, and rollback feasibility before it ships.
Lock duration. The strongest mode held per table, how long it was held, and whether another session waited on it.
Schema coexistence. Whether old instances can still read the new schema shows up here first.
Users notice p99 immediately. Large tables plus frequent schema change.
- peak hold
- 27.4s
- events p99
- 820ms → 6.9s
- waiter
- waiting
Another session was left waiting. events p99 moved 820ms → 6.9s.
Know what happens before you deploy.
Create a disposable production twin for every risky change. Catch migration failures before they reach customers.