Transform reference

Every transform available to a masking rule. The table is generated from the registry in engine/internal/masking/transform.go, so a transform that exists and is not listed here fails the build.

The unique column matters more than it looks. A transform that does not preserve uniqueness cannot be used on a column with a unique constraint: the masked values collide and the update fails partway. af mask plan catches that before anything runs.

Transform	Unique	What it does
address	no	Replaces a street address with a synthetic one of a similar shape.
city	no	Replaces a city with a synthetic one.
company	no	Replaces a company name with a synthetic one that reads as a company.
credit_card	no	Replaces a card number with a Luhn valid test number, so a payment form still validates it and no real card is ever present.
date_shift	no	Moves a date or timestamp by a deterministic offset of up to a year, keeping its format and its time of day.
email	yes	Replaces an address with a unique synthetic one at example.test, which is reserved and can never receive mail.
empty_json	no	Replaces a JSON value with an empty one of the same kind, an object or an array. This is what empties a JSON column that cannot hold null, which nullify cannot do.
first_name	no	Replaces a given name with a synthetic one.
free_text	no	Replaces prose with synthetic prose of a similar length, so a layout built for three paragraphs still gets three paragraphs.
hash_hex	yes	Replaces a value with a keyed hash of the same length. Equality is preserved and nothing else is.
int_fpe	no	Replaces an integer with a different one of the same digit count and sign, so range checks and column widths still hold.
ip	no	Replaces an IP address with one from a documentation range reserved by RFC 5737, which can never route anywhere.
last_name	no	Replaces a family name with a synthetic one.
name	no	Replaces a person’s name with a synthetic one of a similar shape, keeping the number of parts.
nullify	no	Sets the column to null. This is the default for unclassified free text, because a column nobody has confirmed is safe is not safe.
numeric_noise	no	Moves a number by up to ten percent, keeping its sign, scale, and decimal places, so totals stay the right order of magnitude.
phone	no	Replaces the digits of a phone number in place, keeping its length, punctuation, and country prefix so that format checks still pass.
postcode	no	Rewrites a postal code in place, keeping letters as letters and digits as digits so the country’s format still validates.
prefixed_id	yes	Replaces a third party identifier such as cus_ABC123 with one of the same length and prefix, the body being a keyed hash in lowercase hex. Equality and joins survive; the real account it pointed at does not.
preserve	yes	Leaves the value unchanged. Use it to record that a column was reviewed and found safe, rather than leaving it out.
region	no	Replaces a state or subdivision code with a synthetic two letter one. For a column holding the full name of a region, use city or nullify instead.
repository	yes	Replaces an owner/name repository reference with synthetic handles for both halves, the owner masking identically to a username column that shares its link.
string_fpe	no	Replaces a string with one of the same length, keeping digits as digits and letters as letters so a format check still matches.
url	no	Keeps a URL’s scheme and path shape, replacing its host with a synthetic one at example.test.
username	yes	Replaces a handle with a unique synthetic one made of a word and a number.
uuid_remap	yes	Maps a UUID to a different valid UUID. Columns that share a link map identically, so foreign keys still join.

Check constraints

A transform has to satisfy the constraints the column already has.

AF-MSK-004 Masking would violate the check constraint orders_total_positive on
orders.total.
  Next: Choose a format preserving transform for orders.total that satisfies
  orders_total_positive.

numeric_noise keeps a number’s sign and scale and will satisfy most range checks. int_fpe keeps the digit count and sign. A check constraint that encodes a business rule, such as a status being one of five strings, needs preserve rather than a transform: there is no synthetic value that satisfies it and is not the original.

Choosing one

The question is what a test depends on.

A form that validates a card number needs credit_card, which produces a Luhn valid test number. A layout built for three paragraphs needs free_text, which produces three paragraphs. A report that sums a column needs numeric_noise, which keeps totals the right order of magnitude, rather than int_fpe, which does not.

A column that nothing reads can have nullify, and that is the default for unclassified free text on purpose: it makes the absence visible.

nullify cannot empty a column that is NOT NULL, and the commonest shape of free-form column in any schema is jsonb NOT NULL DEFAULT '{}'. That is what empty_json is for: it writes an empty object or an empty array rather than removing the value, so the constraint still holds and a reader that indexes into an array still finds one. It is the default for an unclassified JSON column for the same reason nullify is the default for unclassified text.

Uniqueness

AF-MSK-007 The transform on users.email produced duplicate values under the
unique constraint users_email_key.
  Next: Use a transform that preserves uniqueness, such as email or uuid_remap,
  for users.email.

email, hash_hex, prefixed_id, preserve, repository, username and uuid_remap preserve uniqueness.

name, city, company and the rest do not, because two people can share a name and pretending otherwise would mean generating increasingly unlikely ones to satisfy a constraint the data never had.

The format preserving pair are the ones worth saying twice, because they read like they should be safe here and are not. string_fpe keeps a value’s length and character classes, and int_fpe keeps a number’s digit count and sign, so in both cases two different inputs of the same shape can land on the same output. Keeping a value’s shape says nothing about keeping values apart.

This paragraph said otherwise about both of them, one at a time. It is checked against the registry now, by the same test that generates the table above it, because the table was right the whole time and sitting directly above the sentence contradicting it.

Determinism

Every transform is keyed. The key is generated once and kept, so the same input maps to the same output within a golden and across refreshes: that is what makes link work and two goldens comparable. The key stays inside the boundary the golden is built in.

Related: masking, verification.