Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .changeset/eql-repair-command.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
'stash': minor
'@cipherstash/wizard': patch
---

Add `stash eql repair --drizzle` — repair a migration directory that `drizzle-kit
generate` filled with an un-runnable in-place `ALTER COLUMN … SET DATA TYPE
<eql_v3_*>`, without generating anything (cipherstash/stack#710).

```bash
stash eql repair --drizzle # sweep drizzle/
stash eql repair --drizzle --dry-run # preview; writes nothing
stash eql repair --drizzle --database-url … # leave applied migrations alone
```

Until now the only way to run that sweep was `stash eql migration --drizzle`,
which generates a redundant EQL install migration as a side effect purely to
trigger it — the sweep runs before `drizzle-kit generate` has emitted the broken
statement, so recovery meant creating a migration you did not want. `eql repair`
runs the same rewriter and prints the same report (both commands now share one
reporting path, so the two surfaces cannot drift).

**New: applied-migration awareness.** The sweep has always been unfiltered. That
is harmless for almost every match, because an ALTER to an EQL domain cannot run
— so the migration failed and was never applied. The exception is a `jsonb`
column changed to an EQL domain on an empty table, which applies successfully;
rewriting it afterwards leaves the `.sql` describing a shape the database never
got from it, and a fresh CI or staging database replaying the rewritten file
diverges from the original, silently.

`eql repair` therefore reads `meta/_journal.json` offline and, given
`--database-url` (or `DATABASE_URL`), the latest `created_at` in
`drizzle.__drizzle_migrations`. A migration is applied when its journal `when` is
at or below that watermark — the same timestamp comparison `drizzle-kit migrate`
makes, hashes being written but never compared. Applied migrations are reported
as their own outcome, left untouched, and the command exits non-zero. Without a
database URL the repair proceeds and warns that applied state could not be
verified; if the check is requested but cannot run, nothing is rewritten.

A ledger that is not where the probe looked is reported as **unverified**, not as
"nothing applied" — it means either `drizzle-kit migrate` never ran, or
`drizzle.config.ts` overrode `migrations.table` / `migrations.schema` and the
query went to the wrong relation. `--migrations-table <[schema.]table>` names the
ledger for that case; the value must be a plain (optionally schema-qualified)
identifier and is rejected before connecting otherwise.

An applied migration whose statement the sweep would have skipped regardless — an
undeclared source column, an already-encrypted one, an existing twin — is
reported with that skip reason rather than as an applied-migration refusal.

`rewriteEncryptedAlterColumns` gained `dryRun`, and its `skip` option now accepts
several paths as well as one. The wizard's copy of the rewriter carries the same
change so the two stay in sync; its own sweep is unaffected.
20 changes: 20 additions & 0 deletions packages/cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,26 @@ npx drizzle-kit migrate

---

### `npx stash eql repair --drizzle`

Repairs migrations `drizzle-kit generate` emitted with an in-place `ALTER COLUMN … SET DATA TYPE <eql_v3_*>`, which Postgres cannot run (there is no cast from `text`/`numeric` to an EQL domain). Each is rewritten into an additive `ADD COLUMN "<column>_encrypted"` that preserves the source column.

```bash
npx stash eql repair --drizzle
npx drizzle-kit migrate
```

| Flag | Description |
|------|-------------|
| `--drizzle` | Required. Repair a Drizzle migration directory |
| `--out <path>` | Directory to sweep. Default `drizzle` |
| `--dry-run` | Report what would be rewritten without writing anything |
| `--database-url <url>` | Leave migrations the database has already applied untouched |

This is the same sweep `eql migration --drizzle` performs, without generating an install migration you do not need. With `--database-url` it reads `drizzle.__drizzle_migrations` and refuses to rewrite an already-applied migration — doing so would leave the file describing a shape that database never got from it, and a fresh CI or staging database replaying it would silently diverge. Without a URL it proceeds and warns that applied state could not be verified.

---

## Required database permissions

Before installing EQL, the CLI verifies that the connected role has:
Expand Down
12 changes: 12 additions & 0 deletions packages/cli/src/bin/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ Commands:

eql install Scaffold stash.config.ts (if missing) and install EQL extensions
eql migration Generate an EQL v3 install migration for your ORM (Drizzle)
eql repair Repair migrations with an un-runnable ALTER COLUMN to an encrypted type
eql upgrade Upgrade EQL extensions to the latest version
eql status Show EQL installation status

Expand Down Expand Up @@ -264,6 +265,17 @@ async function runEqlCommand(
})
break
}
case 'repair': {
const { eqlRepairCommand } = await import('../commands/eql/repair.js')
await eqlRepairCommand({
drizzle: flags.drizzle,
out: values.out,
dryRun: flags['dry-run'],
databaseUrl: values['database-url'],
migrationsTable: values['migrations-table'],
})
break
}
case 'upgrade':
await runUpgrade(flags, values)
break
Expand Down
13 changes: 13 additions & 0 deletions packages/cli/src/cli/__tests__/help.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,19 @@ describe('renderCommandHelp', () => {
expect(out).toContain('auth regions')
})

// `eql repair` is the only command that can consult applied state, so its
// --database-url is load-bearing rather than decorative — help must show it
// alongside the flags that shape the sweep.
it('renders every flag of the standalone repair command', () => {
const out = renderCommandHelp('eql repair', RUNNER)
expect(out).not.toBeNull()
expect(out).toContain('Usage: npx stash eql repair [options]')
expect(out).toContain('--drizzle')
expect(out).toContain('--out <path>')
expect(out).toContain('--dry-run')
expect(out).toContain('--database-url <url>')
})

it('renders a summary-only command without empty Options/Examples', () => {
const out = renderCommandHelp('wizard', RUNNER)
expect(out).toContain('Usage: npx stash wizard [options]')
Expand Down
48 changes: 48 additions & 0 deletions packages/cli/src/cli/registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,54 @@ export const registry: CommandGroup[] = [
DRY_RUN_FLAG,
],
},
{
name: 'eql repair',
summary:
'Repair migrations drizzle-kit generated with an un-runnable ALTER COLUMN to an encrypted type',
long: [
'Sweep an existing Drizzle output directory for in-place',
'`ALTER COLUMN ... SET DATA TYPE <eql domain>` statements — which cannot run,',
'because Postgres has no cast from text/numeric to an EQL domain — and rewrite',
'each into an additive encrypted column that preserves the source column.',
'',
'This is the same sweep `eql migration --drizzle` runs, without having to',
'generate an EQL install migration you do not want just to trigger it.',
'',
'Migrations the database has already applied are reported and left alone:',
'rewriting one would leave its .sql describing a shape that database never got',
'from it, so a fresh CI or staging database replaying the file would silently',
'diverge. Pass --database-url so that check can run; without it the repair',
'proceeds and warns that applied state could not be verified. If your',
'drizzle.config.ts overrides `migrations.table` / `migrations.schema`, name',
'the ledger with --migrations-table — otherwise the check queries the default',
'relation, finds nothing, and reports applied state as unverified.',
].join('\n'),
examples: [
'eql repair --drizzle',
'eql repair --drizzle --dry-run',
'eql repair --drizzle --out db/migrations --database-url postgres://…',
],
flags: [
{
name: '--drizzle',
description: 'Repair a Drizzle migration directory.',
},
{
name: '--out',
value: '<path>',
description:
'Directory holding the migrations to sweep. Defaults to `drizzle`; set it to match your drizzle.config.ts.',
},
{
name: '--migrations-table',
value: '<[schema.]table>',
description:
"Drizzle's migration ledger, when drizzle.config.ts overrides `migrations.table` / `migrations.schema`. Defaults to `drizzle.__drizzle_migrations`. Only read with --database-url.",
},
DRY_RUN_FLAG,
DATABASE_URL_FLAG,
],
},
{
name: 'eql upgrade',
summary: 'Upgrade EQL extensions to the latest version',
Expand Down
18 changes: 15 additions & 3 deletions packages/cli/src/commands/db/rewrite-migrations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1038,10 +1038,19 @@ interface RewriteSweepError extends Error, PartialRewriteResult {}
* shipping broken SQL. Statements sitting inside a
* SQL comment — or inside a single-quoted string literal, where they are data
* rather than SQL — are inert and are neither rewritten nor reported.
*
* `options.skip` names files to leave on disk — they are still READ, because a
* column's current type comes from the whole corpus, but never written. One
* path or many: `eql migration` passes the install migration it just generated,
* `eql repair` passes every migration already applied to the database.
*
* `options.dryRun` computes the identical result without writing anything, so a
* caller can preview a repair — or ask which files WOULD be rewritten — before
* touching migrations that are about to be applied.
*/
export async function rewriteEncryptedAlterColumns(
outDir: string,
options: { skip?: string } = {},
options: { skip?: string | readonly string[]; dryRun?: boolean } = {},
): Promise<RewriteResult> {
const entries = await readdir(outDir).catch(
(error: NodeJS.ErrnoException) => {
Expand All @@ -1057,6 +1066,9 @@ export async function rewriteEncryptedAlterColumns(
const staged: StagedColumn[] = []
const seen = new Set<string>()
const stagedTargets = new Set<string>()
const skipFiles = new Set(
typeof options.skip === 'string' ? [options.skip] : (options.skip ?? []),
)

/** Record a skip once — the strict pass and the broad scan can both find it. */
const skip = (file: string, statement: string, reason: SkipReason): void => {
Expand Down Expand Up @@ -1084,7 +1096,7 @@ export async function rewriteEncryptedAlterColumns(

try {
for (const [filePath, original] of contents) {
if (options.skip && filePath === options.skip) continue
if (skipFiles.has(filePath)) continue

// Reset the regex's lastIndex — it's stateful on /g
ALTER_COLUMN_TO_ENCRYPTED_RE.lastIndex = 0
Expand Down Expand Up @@ -1165,7 +1177,7 @@ export async function rewriteEncryptedAlterColumns(
)

if (updated !== original) {
await writeFile(filePath, updated, 'utf-8')
if (!options.dryRun) await writeFile(filePath, updated, 'utf-8')
rewritten.push(filePath)
staged.push(...fileStaged)
}
Expand Down
Loading
Loading