Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .changeset/remove-eql-v2-migrate-classifier.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
'@cipherstash/migrate': patch
---

Drop EQL v2 from the domain-type classifier. `classifyEqlDomain` (and the
`detectColumnEqlVersion` / `listEncryptedColumns` / `resolveEncryptedColumn`
resolution built on it) no longer recognise the legacy `eql_v2_encrypted`
domain — v3 is the sole generation this workspace authors and backfills, so a
column's version is now determined solely from its self-describing `eql_v3_*`
domain type. A legacy v2 column's version is carried by the manifest's recorded
`eqlVersion` instead (the CLI's `encrypt status` / `status` renderers already
fall back to it), so status output is unchanged for v2 columns already recorded
in `.cipherstash/migrations.json`. A v2 column backfilled from here on records
no `eqlVersion` and so reports no version in `stash encrypt status` — the v2
lifecycle itself (cut-over, then dropping `<column>_plaintext`) is unaffected.

This removes v2 *classification*, not the v2 read path: existing v2 ciphertext
remains decryptable through `@cipherstash/stack`. `EqlVersion` keeps its `2`
member for manifest-sourced legacy values; the exported function signatures are
unchanged.
12 changes: 8 additions & 4 deletions packages/cli/src/commands/encrypt/__tests__/encrypt-v3.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -430,10 +430,14 @@ describe('encrypt drop — EQL version awareness', () => {
// After cutover renamed the ciphertext onto `email`, no counterpart is
// resolvable BY DESIGN. The fail-closed guard must recognize this state
// rather than blocking the one drop the lifecycle actually wants.
lifecycleMock.mockResolvedValue({
info: null,
candidates: [{ column: 'email', domain: 'eql_v2_encrypted', version: 2 }],
})
//
// `candidates` is EMPTY, not `[{ column: 'email', version: 2 }]`: the
// classifier no longer recognises `eql_v2_encrypted`, so a post-cutover v2
// column drops out of `listEncryptedColumns` entirely. The state reaches
// `explainUnresolved` as "no EQL columns at all", which is exactly the
// case it already falls through on. This pins that the v2 lifecycle still
// works through the narrower classifier.
lifecycleMock.mockResolvedValue({ info: null, candidates: [] })
migrateMocks.progress.mockResolvedValueOnce({ phase: 'cut-over' })

await dropCommand({ table: 'users', column: 'email' })
Expand Down
20 changes: 15 additions & 5 deletions packages/cli/src/commands/encrypt/backfill.ts
Original file line number Diff line number Diff line change
Expand Up @@ -141,8 +141,11 @@ export async function backfillCommand(options: BackfillCommandOptions) {
// v2 or v3 changes the rest of the LIFECYCLE (v3 has no cut-over — the
// ladder is backfill → switch-by-name → drop), so detect it up front,
// record it in the manifest, and tell the user which path they're on.
// `null` means the target column doesn't exist or isn't an EQL domain —
// let the existing checks below produce their specific errors.
// `null` means the target column doesn't exist, isn't an EQL domain, or is
// a legacy `eql_v2_encrypted` column (no longer classified — v3 is the sole
// authored generation). Every one of those falls through to the v2 ladder,
// which is the correct default for the v2 case and lets the existing checks
// below produce their specific errors for the other two.
const eqlVersion = await detectColumnEqlVersion(
db,
options.table,
Expand Down Expand Up @@ -553,12 +556,19 @@ function buildManifestEntry(
// convention only, never relied upon.
encryptedColumn,
// v2's ladder ends with the rename cut-over; v3 has none — its end
// state is the plaintext column dropped.
// state is the plaintext column dropped. An unclassified column (null,
// which now includes a legacy v2 domain) takes the v2 ladder.
targetPhase: eqlVersion === 3 ? 'dropped' : 'cut-over',
}
if (pkColumn) entry.pkColumn = pkColumn
// Absent means UNKNOWN (detection couldn't see the column), not v2 —
// readers fall back to the domain type in the database.
// Absent means the classifier returned null: the column isn't there, isn't
// an EQL domain, or carries the legacy `eql_v2_encrypted` domain (which
// `classifyEqlDomain` no longer recognises — v3 is the sole authored
// generation). Readers fall back to the live domain type, which yields null
// for those same three cases, so `encrypt status` reports no version rather
// than guessing one. Manifests written before v2 classification was dropped
// still carry `eqlVersion: 2`, so existing v2 columns keep their version;
// only a v2 column backfilled from here on records none.
if (eqlVersion) entry.eqlVersion = eqlVersion
return entry
}
Expand Down
6 changes: 4 additions & 2 deletions packages/cli/src/commands/encrypt/cutover.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,10 @@ export async function cutoverCommand(options: CutoverCommandOptions) {
// Fail closed on ambiguity: `info === null` with EQL columns present
// means we can't tell WHICH lifecycle applies — running the v2 config
// machine against (possibly) v3 columns would only produce a misleading
// downstream error. (No EQL columns at all, or the post-cutover v2
// same-name state, still falls through to the v2 preconditions below.)
// downstream error. (A table with no EQL v3 columns still falls through to
// the v2 preconditions below — which now also covers the post-cutover v2
// same-name state, since an `eql_v2_encrypted` column is no longer
// classified and so never appears as a candidate.)
const unresolved = explainUnresolved(
options.table,
options.column,
Expand Down
5 changes: 3 additions & 2 deletions packages/cli/src/commands/encrypt/drop.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,9 @@ export async function dropCommand(options: DropCommandOptions) {
// the wrong ciphertext and generate an irreversible drop of the wrong
// data. (The post-cutover v2 state — `<col>` itself carries the v2
// domain, counterpart legitimately unresolvable — falls through to the
// v2 path; live truth from the DB wins over the manifest's cached
// version throughout.)
// v2 path: the classifier recognises `eql_v3_*` only, so that column is
// not a candidate and the table reads as having no EQL columns. Live
// truth from the DB wins over the manifest's cached version throughout.)
const unresolved = explainUnresolved(
options.table,
options.column,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/**
* Pins {@link explainUnresolved}'s fail-closed contract now that the domain
* classifier recognises `eql_v3_*` only.
*
* `listEncryptedColumns` can no longer emit `version: 2` — a legacy
* `eql_v2_encrypted` column is not classified as an EQL column at all, so it
* never reaches this function as a candidate. The post-cutover v2 state (the
* ciphertext renamed onto the plaintext column's own name) therefore arrives
* here as an EMPTY candidate list, which the first guard already falls through
* on. These tests exist so removing the now-unreachable `version === 2` branch
* is provably behaviour-preserving, and so a future v2 sweep cannot delete the
* empty-list guard the v2 lifecycle actually depends on.
*/

import type { EncryptedColumnInfo } from '@cipherstash/migrate'
import { describe, expect, it } from 'vitest'
import { explainUnresolved } from '../resolve-eql.js'

const v3 = (
column: string,
domain = 'eql_v3_text_eq',
): EncryptedColumnInfo => ({
column,
domain,
version: 3,
})

describe('explainUnresolved', () => {
it('falls through (null) when the table has no EQL columns at all', () => {
// Both the not-yet-backfilled case and the post-cutover v2 same-name case
// land here: the caller's own preconditions produce the accurate error.
expect(explainUnresolved('users', 'email', [])).toBeNull()
})

it('fails closed, naming every candidate, when none is identifiable', () => {
const message = explainUnresolved('users', 'email', [
v3('a_enc'),
v3('b_enc', 'eql_v3_text_search'),
])

expect(message).toContain('Cannot identify which encrypted column')
expect(message).toContain('a_enc (eql_v3_text_eq)')
expect(message).toContain('b_enc (eql_v3_text_search)')
expect(message).toContain('--encrypted-column')
})

it('gives no free pass to a candidate sharing the plaintext column name', () => {
// The removed branch exempted a SAME-NAME candidate, but only at
// `version === 2`. A v3 domain on the plaintext column's own name is not
// the post-cutover state (v3 has no cut-over rename), so it must still
// fail closed rather than let a destructive command guess a lifecycle.
const message = explainUnresolved('users', 'email', [
v3('email'),
v3('email_enc'),
])

expect(message).toContain('Cannot identify which encrypted column')
})
})
24 changes: 13 additions & 11 deletions packages/cli/src/commands/encrypt/lib/resolve-eql.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,26 +68,28 @@ export async function resolveColumnLifecycle(

/**
* Explain a failed resolution (`info === null`) to the user, or return
* `null` when the failure is fine to fall through to the v2 lifecycle:
* `null` when the failure is fine to fall through to the v2 lifecycle.
*
* - No EQL columns at all → the v2 phase/config preconditions produce the
* accurate error ("not backfilled", "no pending config", …).
* - The plaintext column ITSELF carries the v2 domain → the normal
* post-cutover v2 state (`<col>` was renamed onto the ciphertext), where
* "no counterpart" is expected, not a problem.
* The one fall-through case is "no EQL columns at all", which the v2
* phase/config preconditions turn into an accurate error ("not backfilled",
* "no pending config", …). Since `classifyEqlDomain` recognises `eql_v3_*`
* only, that case now also covers the post-cutover v2 state — `<col>` was
* renamed onto the ciphertext, and its `eql_v2_encrypted` domain is no longer
* classified, so the column never appears as a candidate. (It used to arrive
* here as a `version: 2` candidate and needed its own exemption.)
*
* Anything else means EQL columns exist but none is identifiable — the
* caller must fail closed with this message rather than guess a lifecycle.
* A non-empty candidate list therefore means EQL v3 columns exist but none is
* identifiable — the caller must fail closed with this message rather than
* guess a lifecycle, including when one candidate happens to share the
* plaintext column's name (v3 has no cut-over rename, so that is not the
* post-cutover state).
*/
export function explainUnresolved(
table: string,
column: string,
candidates: readonly EncryptedColumnInfo[],
): string | null {
if (candidates.length === 0) return null
if (candidates.some((c) => c.column === column && c.version === 2)) {
return null
}
const listed = candidates
.map((c) => ` - ${c.column} (${c.domain})`)
.join('\n')
Expand Down
34 changes: 25 additions & 9 deletions packages/migrate/src/__tests__/version.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,13 @@ function mockClient(rows: Array<Record<string, unknown>>) {
}

describe('classifyEqlDomain', () => {
it('maps eql_v2_encrypted to 2', () => {
expect(classifyEqlDomain('eql_v2_encrypted')).toBe(2)
it('no longer classifies eql_v2_encrypted — v3 is the sole authored generation', () => {
// The v2 branch was removed: this workspace authors/backfills v3 only, so
// the domain classifier recognises `eql_v3_*` alone. A legacy v2 column's
// version now comes from the manifest's recorded `eqlVersion`, not here.
// (Existing v2 ciphertext stays decryptable — only classification of v2 as
// an authorable generation is dropped.)
expect(classifyEqlDomain('eql_v2_encrypted')).toBeNull()
})

it('maps any eql_v3_* domain to 3', () => {
Expand All @@ -46,10 +51,20 @@ describe('classifyEqlDomain', () => {

describe('detectColumnEqlVersion', () => {
it('classifies from the domain type', async () => {
const { client } = mockClient([{ domain_name: 'eql_v2_encrypted' }])
const { client } = mockClient([{ domain_name: 'eql_v3_text_search' }])
expect(
await detectColumnEqlVersion(client, 'users', 'email_encrypted'),
).toBe(2)
).toBe(3)
})

it('returns null for a legacy eql_v2_encrypted domain (v2 no longer classified)', async () => {
// A v2 column still exists physically, but the classifier no longer treats
// it as an authorable EQL generation — callers fall back to the manifest's
// recorded eqlVersion.
const { client } = mockClient([{ domain_name: 'eql_v2_encrypted' }])
expect(
await detectColumnEqlVersion(client, 'users', 'ssn_encrypted'),
).toBeNull()
})

it('returns null for a plaintext column (base type, not a domain)', async () => {
Expand Down Expand Up @@ -88,16 +103,17 @@ describe('detectColumnEqlVersion', () => {
})

describe('listEncryptedColumns', () => {
it('returns only EQL-domain columns, classified', async () => {
it('returns only EQL v3-domain columns, classified (legacy v2 domains excluded)', async () => {
const { client } = mockClient([
{ column: 'id', domain_name: 'int8' },
{ column: 'email', domain_name: 'text' },
{ column: 'email_enc', domain_name: 'eql_v3_text_search' },
// A legacy v2 column is no longer classified as EQL, so it drops out of
// the encrypted-column listing entirely.
{ column: 'ssn_encrypted', domain_name: 'eql_v2_encrypted' },
])
expect(await listEncryptedColumns(client, 'users')).toEqual([
{ column: 'email_enc', domain: 'eql_v3_text_search', version: 3 },
{ column: 'ssn_encrypted', domain: 'eql_v2_encrypted', version: 2 },
])
})
})
Expand Down Expand Up @@ -153,10 +169,10 @@ describe('pickEncryptedColumn', () => {
})

it('never resolves the plaintext column to itself', () => {
// Post-cutover v2: `email` itself carries the v2 domain. It is the
// ciphertext, not a counterpart of itself.
// A column that IS the plaintext argument cannot be its own encrypted
// counterpart, even when it's the table's sole EQL-domain column.
expect(
pickEncryptedColumn([col('email', 'eql_v2_encrypted', 2)], 'email'),
pickEncryptedColumn([col('email', 'eql_v3_encrypted')], 'email'),
).toBeNull()
})

Expand Down
15 changes: 11 additions & 4 deletions packages/migrate/src/version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ export type EqlVersion = 2 | 3
export interface EncryptedColumnInfo {
/** The column's name, exactly as Postgres reports it. */
column: string
/** The EQL domain name, e.g. `eql_v2_encrypted` or `eql_v3_text_search`. */
/** The EQL domain name, e.g. `eql_v3_text_search` or `eql_v3_integer_ord`. */
domain: string
version: EqlVersion
}
Expand All @@ -30,12 +30,19 @@ export interface EncryptedColumnInfo {
* rule lives, and why detection never relies on column NAMES: the
* `<column>_encrypted` naming is a convention, neither enforced nor required.
*
* - `eql_v2_encrypted` → 2
* - `eql_v3_*` (e.g. `eql_v3_text_search`, `eql_v3_integer_ord`) → 3
* - anything else → `null` (not an EQL column)
* - anything else → `null` (not an EQL v3 column)
*
* v3 is the sole generation this workspace authors and backfills, so the
* classifier only recognises `eql_v3_*` domains. A legacy `eql_v2_encrypted`
* column is therefore no longer classified here — its version is carried by
* the manifest's recorded `eqlVersion` instead (see the `?? eqlVersion`
* fallbacks in the CLI's `encrypt status` / `status` renderers). Existing v2
* ciphertext stays decryptable; only its *classification as an authorable
* generation* is dropped. `EqlVersion` keeps the `2` member for those
* manifest-sourced legacy values.
*/
export function classifyEqlDomain(domain: string): EqlVersion | null {
if (domain === 'eql_v2_encrypted') return 2
// Underscore included: a bare `startsWith('eql_v3')` would also claim
// hypothetical future generations like `eql_v30_*`.
if (domain.startsWith('eql_v3_')) return 3
Expand Down