Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 36 additions & 5 deletions docs/staff-kyc-clearance.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,43 @@ be reviewed and reproducible like any other schema/data change.

1. Open a PR against the api repo that adds a new migration under `migration/` (never modify an
existing one — `api-migration-check.yaml` blocks that). The migration runs a guarded, idempotent
`UPDATE user_data SET "verifiedName" = … WHERE id = … AND "verifiedName" IS NULL`.
`UPDATE user_data SET "verifiedName" = … WHERE id = … AND BTRIM(COALESCE("verifiedName", ''), <BlankChars>) = ''`.
The closing assertion must check the **clearance predicate itself** — that
`… WHERE id = … AND BTRIM("verifiedName", <BlankChars>) <> ''` yields exactly one row — and never
equality with the supplied name. The goal state is a cleared account, not a particular spelling, and
an equality assertion fails whichever precondition it is paired with: against a blankness
precondition (the shape of the first such migration) it throws when an identity-verified path wrote a
different but perfectly valid name in the meantime; against its own negation it does the opposite and
silently overwrites that name.

The precondition must then be the **exact negation** of that assertion **including the NULL case** —
`BTRIM(COALESCE("verifiedName", ''), <BlankChars>) = ''`, not
`BTRIM("verifiedName", <BlankChars>) = ''`. Without the `COALESCE`, NULL yields NULL rather than
true, so the ordinary un-backfilled account is neither repaired nor accepted. Use the same
`BlankChars` set as `StaffKycClearanceService`. A narrower `"verifiedName" IS NULL` precondition has
the mirror-image flaw: it leaves a present-but-blank name (a lone tab, a non-breaking space) as a
state the migration refuses to repair and then refuses to accept.

The `UPDATE` must be coupled to a durable before/after audit row — a `log` insert (`system` `'User'`,
`subsystem` `'StaffVerifiedNameBackfill'`, `severity` `'Info'`) in the same statement via a
data-modifying CTE, with the update conditioned on `EXISTS (SELECT 1 FROM "audit")` so the column
cannot change unaudited. `verifiedName` is PII; CONTRIBUTING treats unaudited mutation of it as
blocking.

Every one of these mistakes is boot-fatal rather than merely wrong: `migrationsTransactionMode`
defaults to `all`, so the throw rolls back the whole release's migration batch and fails
`DataSource.initialize()`.

2. **A real person's name is PII and must not be hard-coded in this public repo.** The migration
reads the value from a deployment secret (e.g. `process.env.STAFF_VERIFIED_NAME_<id>`), set in the
production config, and applies it only when present. A non-personal service designation (for a
machine account that cannot complete a personal identification) is not PII and may appear inline.
The concrete name↔account mapping is recorded in the private operations repo, not here.
reads the value from a deployment variable (e.g. `process.env.STAFF_VERIFIED_NAME_<id>`), set in the
production config. On PRD the variable is **mandatory**: the migration throws when it is absent,
rather than silently recording a no-op — so the value has to be live in the production environment
before the migration is merged to `develop`. `auto-release-pr.yaml` keeps a `develop` → `main`
release PR open continuously — every `develop` push either opens one or lands on the one already
open — so `develop` is the last point at which the order can still be arranged. A non-personal
service designation (for a machine account that cannot complete a personal
identification) is not PII and may appear inline. The concrete name↔account mapping is recorded in
the private operations repo, not here.
3. Merge the PR through the normal review, then release `develop → main`. The production deploy runs
pending migrations automatically when `SQL_MIGRATE=true` (see `migrationsRun` in
`src/config/config.ts`) — no shell access to the database is involved.
Expand Down
101 changes: 101 additions & 0 deletions migration/1785635000000-BackfillDebugStaffVerifiedName.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
/**
* @typedef {import('typeorm').MigrationInterface} MigrationInterface
* @typedef {import('typeorm').QueryRunner} QueryRunner
*/

// Same character set as `BlankChars` in StaffKycClearanceService — every character
// `String.prototype.trim()` strips. Postgres' bare `TRIM(x)` removes ASCII space only, so a name of a
// single tab or a non-breaking space would pass a `TRIM(x) <> ''` test while the clearance query still
// rejects it. Duplicated rather than imported: migrations are plain JS executed by TypeORM and cannot
// pull in application sources.
const BLANK_CHARS =
'\u0009\u000a\u000b\u000c\u000d\u0020\u00a0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007' +
'\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000\ufeff';

/**
* PRD-only backfill for the remaining staff account gated out by the staff-clearance rule (#4395 →
* #4572). The earlier backfill (#4574) covered two accounts; user data 403938 (the Debug account) was
* not among them and still fails every elevated endpoint with STAFF_KYC_REQUIRED.
*
* No plaintext personal name lives in this file: the value is read from the deployment variable
* STAFF_VERIFIED_NAME_403938, which is mandatory on PRD so TypeORM cannot record a partial/no-op
* migration when it is missing. The update is idempotent (only touches a null-or-blank verifiedName)
* and coupled to a durable before/after audit entry. Guarded to prd; a no-op elsewhere.
*
* The closing assertion checks the clearance predicate itself rather than equality with the supplied
* name: should an identity-verified path have written a different (correct) name in the meantime, that
* account is cleared and the migration must not fail the deploy over the spelling. That divergence is
* not silent — it is recorded as its own audit entry, so the deployed state never differs from the
* reviewed one without a trace.
* @class @implements {MigrationInterface}
*/
module.exports = class BackfillDebugStaffVerifiedName1785635000000 {
name = 'BackfillDebugStaffVerifiedName1785635000000';

async up(queryRunner) {
if (process.env.ENVIRONMENT !== 'prd') return;

const verifiedName = process.env.STAFF_VERIFIED_NAME_403938?.trim();
if (!verifiedName) throw new Error('STAFF_VERIFIED_NAME_403938 is required for the PRD staff-name backfill');

// `needsBackfill` is the exact negation of the closing assertion below. The two must stay
// complementary: a precondition of `verifiedName IS NULL` against a non-blank postcondition would
// leave a present-but-blank name (a lone tab, a non-breaking space) as a state the migration
// refuses to repair and then refuses to accept — and because `migrationsTransactionMode` defaults
// to 'all', that throw rolls back the whole release's batch and takes the boot down with it.
//
// `noteworthy` is what gets audited: the repair itself, or the deliberate decision to keep a
// divergent name that an identity-verified path wrote in the meantime. A re-run after a successful
// backfill is neither, so it stays a true no-op instead of appending an audit row every time.
// Array.of avoids looking like MSSQL bracket quoting to the repository's migration syntax guard.
await queryRunner.query(
`WITH "target" AS (
SELECT "id",
"verifiedName" AS "previousVerifiedName",
BTRIM(COALESCE("verifiedName", ''), $2::varchar) = '' AS "needsBackfill"
FROM "user_data"
WHERE "id" = 403938
FOR UPDATE
),
"noteworthy" AS (
SELECT "id", "previousVerifiedName", "needsBackfill"
FROM "target"
WHERE "needsBackfill" OR "previousVerifiedName" IS DISTINCT FROM $1::varchar
),
"audit" AS (
INSERT INTO "log" ("created", "updated", "system", "subsystem", "severity", "message")
SELECT now(), now(), 'User', 'StaffVerifiedNameBackfill', 'Info',
json_agg(json_build_object(
'userDataId', "id",
'previousVerifiedName', "previousVerifiedName",
'nextVerifiedName', CASE WHEN "needsBackfill" THEN $1::varchar ELSE "previousVerifiedName" END,
'action', CASE WHEN "needsBackfill" THEN 'backfilled' ELSE 'keptExistingName' END
) ORDER BY "id")::text
FROM "noteworthy"
HAVING count(*) > 0
RETURNING 1
)
UPDATE "user_data" ud
SET "verifiedName" = $1::varchar, "updated" = now()
FROM "target" t
WHERE ud."id" = t."id" AND t."needsBackfill" AND EXISTS (SELECT 1 FROM "audit")`,
Array.of(verifiedName, BLANK_CHARS),
);

const rows = await queryRunner.query(
`SELECT count(*)::int AS "clearedCount"
FROM "user_data"
WHERE "id" = 403938 AND BTRIM("verifiedName", $1::varchar) <> ''`,
Array.of(BLANK_CHARS),
);

if (Number(rows.at(0)?.clearedCount) !== 1) {
throw new Error('PRD staff-name backfill did not reach the required state for user data 403938');
}
}

async down() {
// No-op: a granted clearance is not auto-revoked here; removal requires a separate reviewed,
// audited revocation so an unrelated rollback cannot silently erase an identity grant.
}
};
Loading