fix(cli): mixed-table encrypt lifecycle, and a stash init scaffold that compiles (#772 review findings 6, 7)

.changeset/encrypt-client-guard-parity.md

@@ -0,0 +1,20 @@
+---
+'stash': patch
+---
+
+`stash encrypt backfill` now names the cause when the encryption client has no
+initialized encrypt config, instead of reporting a missing table.
+
+The guard that refuses an unusable client file existed twice — once in
+`loadEncryptConfig` (`stash db push` / `db validate`) and once, hand-copied, in
+`loadEncryptionContext` (`stash encrypt backfill`). The copies had already
+drifted: for a client whose `getEncryptConfig()` returns nothing, `db push`
+exited 1 with `Encryption client in <file> has no initialized encrypt config`,
+while `encrypt backfill` fell through to `Table "users" was not found in the
+encryption client exports. Available: (none)` — naming the symptom rather than
+the cause, which is precisely the failure the guard was added to eliminate.
+
+Both loaders now call one shared guard, so a single file cannot produce two
+different diagnoses, and the refusals are pinned at both public entry points.
+The un-replaced `stash init` scaffold is unaffected — it was already refused by
+both, with the same message.

.changeset/encrypt-lifecycle-mixed-table.md

@@ -0,0 +1,59 @@
+---
+'stash': patch
+---
+
+`stash encrypt cutover` and `stash encrypt drop` no longer act on — or report
+success for — an encrypted column they only guessed at.
+
+On a table holding both a legacy EQL v2 pair (`ssn` / `ssn_encrypted`) and one
+unrelated EQL v3 column, the v2 ciphertext column is not classified as an EQL
+column at all, so column resolution fell through to the "this is the table's
+only EQL column" rule and claimed the unrelated v3 column. Three consequences,
+all now fixed:
+
+- **`cutover` reported success for work it never did.** Its EQL v3 branch had no
+  guard on how the column was resolved, and returned without setting an exit
+  code — so it printed "point your application at `email_enc`" and exited 0
+  while the v2 rename never ran. A scripted rollout read that as complete. It
+  now refuses, and exits 1, exactly as `drop` already did.
+
+- **The recorded pairing was discarded.** `encrypt backfill` writes the true
+  `encryptedColumn` to `.cipherstash/migrations.json`, so the answer was already
+  on disk — but a hint that failed to resolve was dropped entirely and the
+  re-resolution reached the guess. A hint naming a column that still exists but
+  is not an EQL v3 column is now reported as what it is (most often a legacy
+  `eql_v2_encrypted` counterpart) instead of being replaced by a guess. A
+  genuinely stale hint — one naming a column that is gone — still falls back to
+  the naming convention as before.
+
+- **`drop`'s refusal message prescribed the guess.** It told the user to re-run
+  `backfill --encrypted-column <the guessed column>`. Following it recorded the
+  guess as fact, so the next run resolved "by hint", walked past the refusal,
+  and passed the coverage check vacuously — an unrelated but legitimately
+  backfilled column is non-NULL on every row — then generated a live
+  `DROP COLUMN` on the plaintext and exited 0. The message now asks for the
+  column that actually encrypts the named one, and says explicitly not to record
+  the guess.
+
+The `unresolvedHint` refusal is scoped to tables that actually hold EQL v3
+columns a guess could wrongly claim. A **pure-v2** table has none, so it still
+falls through to the EQL v2 lifecycle exactly as before — including when
+`encrypt backfill` recorded an `encryptedColumn` for it, which it does for v2
+columns too.
+
+Two cases DO newly exit 1, both deliberately:
+
+- Any table with at least one EQL v3 column where the manifest records an
+  `encryptedColumn` that exists but is not one of them — not only the
+  v2-pair-plus-one-v3 shape. The recorded pairing is authoritative and
+  disagrees with every candidate, so guessing past it is the bug.
+
+- A column whose encrypted counterpart could only be identified **by
+  elimination** — no recorded `encryptedColumn`, no `<col>_encrypted` name
+  match, one EQL column left once the plaintext column itself is excluded.
+  `cutover` now refuses this as `drop` already did. Note `.cipherstash/` is
+  gitignored, so `migrations.json` is machine-local: a fresh clone or CI runner
+  can hit this on a **pure-v3** table whose encrypted column is named
+  unconventionally, where `cutover` previously exited 0 with "not applicable".
+  Re-run `stash encrypt backfill --table T --column C --encrypted-column <name>`
+  to record the pairing.

.changeset/init-scaffold-compiles.md

@@ -0,0 +1,27 @@
+---
+'stash': patch
+---
+
+The client file `stash init` writes now compiles.
+
+Both placeholder templates emitted `await Encryption({ schemas: [] })`, and
+`Encryption` requires at least one table — an empty schema set is a deliberate
+compile error, so it cannot be relaxed. Every `stash init` therefore left a
+project whose first `tsc` or `next build` failed, in a file the CLI had just
+told the user not to hand-edit. (The previous scaffold called `EncryptionV3`,
+whose looser bound accepted `[]`; collapsing that into an alias of `Encryption`
+tightened it.)
+
+The scaffold now declares a single sentinel table, `__stash_placeholder__`, so
+the file typechecks as written. Every command that reads the encryption client
+— `stash db push`, `stash db validate`, and `stash encrypt backfill` — refuses
+to run while that table is still the only one declared, and names it, rather
+than failing later with a confusing "table not found". (`stash encrypt cutover`
+and `stash encrypt drop` do not read the client file at all; they resolve
+against the database.)
+
+Nothing in the repo compiled this output before: `packages/cli` has no
+typecheck step, the codegen tests only string-match fragments of the template,
+and the step test stubs the generator out entirely. Both templates are now
+committed as fixtures that CI typechecks, pinned byte-for-byte to the generator
+so they cannot drift.

.changeset/migrate-column-exists.md

@@ -0,0 +1,21 @@
+---
+'@cipherstash/migrate': minor
+'stash': patch
+---
+
+Add `columnExists(client, tableName, columnName)` — a case-exact "does this
+column exist at all?" catalog probe, distinct from `detectColumnEqlVersion`'s
+"and is it an EQL column?".
+
+Callers need that difference to tell a STALE column reference (it is gone) from
+a live one the domain classifier simply does not recognise — most often a legacy
+`eql_v2_encrypted` counterpart.
+
+`stash encrypt cutover` / `drop` had a private copy of this probe built on a bare
+`to_regclass($1)`. That form *parses* its argument and case-folds unquoted
+identifiers, so on a Prisma-style `"User"` table it resolved `user`, reported the
+column missing, and treated a valid recorded pairing as stale — silently skipping
+the fail-closed that stops those commands acting on a guessed encrypted column.
+The shared implementation quotes with `format('%I')` first, like every other
+catalog probe in this package, so the lookup is case-exact while still honouring
+`search_path` for unqualified names.

.github/workflows/integration-drizzle.yml

@@ -153,8 +153,14 @@ jobs:
       #
       # Step env, not a `.env` file: `dotenv/config` does not override an
       # already-set `process.env`, so these win and no secret is written to disk.
+      # Through turbo, not a bare `pnpm --filter`: `test:integration` declares
+      # `dependsOn: ["^build", "build"]`, and the only build in this job is the
+      # setup action's `--filter stash`, which reaches these packages purely by
+      # coincidence of stash's own dependency graph. `--env-mode=loose` because
+      # turbo defaults to strict and would otherwise withhold the step env below
+      # from the test process (#787 review follow-up).
       - name: Drizzle v3 integration suites
-        run: pnpm --filter @cipherstash/stack-drizzle run test:integration
+        run: pnpm exec turbo run test:integration --filter @cipherstash/stack-drizzle --env-mode=loose
         env:
           DATABASE_URL: ${{ steps.db.outputs.database-url }}
           PGRST_URL: ${{ steps.db.outputs.pgrest-url }}
@@ -164,7 +170,7 @@ jobs:
       # `isInstalled` short-circuits against the DB the first invocation already
       # provisioned — a fast no-op check, not a second schema apply.
       - name: Shared core + identity + wasm integration suites
-        run: pnpm --filter @cipherstash/stack run test:integration
+        run: pnpm exec turbo run test:integration --filter @cipherstash/stack --env-mode=loose
         env:
           DATABASE_URL: ${{ steps.db.outputs.database-url }}
           PGRST_URL: ${{ steps.db.outputs.pgrest-url }}

.github/workflows/integration-prisma-next.yml

@@ -114,7 +114,7 @@ jobs:
       # `stash eql install --eql-version 3`, so an installer regression fails
       # here rather than hiding behind a test-only SQL apply.
       - name: prisma-next v3 family suites
-        run: pnpm --filter @cipherstash/prisma-next run test:integration
+        run: pnpm exec turbo run test:integration --filter @cipherstash/prisma-next --env-mode=loose
         env:
           # Step env, not a `.env` file: `dotenv/config` does not override an
           # already-set `process.env`, so these win and no secret hits disk.

.github/workflows/integration-supabase.yml

@@ -129,7 +129,7 @@ jobs:
       # Step env, not a `.env` file: `dotenv/config` does not override an
       # already-set `process.env`, so these win and no secret is written to disk.
       - name: Supabase v3 integration suites
-        run: pnpm --filter @cipherstash/stack-supabase run test:integration
+        run: pnpm exec turbo run test:integration --filter @cipherstash/stack-supabase --env-mode=loose
         env:
           DATABASE_URL: ${{ steps.db.outputs.database-url }}
           PGRST_URL: ${{ steps.db.outputs.pgrest-url }}
@@ -139,7 +139,7 @@ jobs:
       # `isInstalled` short-circuits against the DB the first invocation already
       # provisioned — so this is a fast no-op check, not a second schema apply.
       - name: Shared core integration suites (against Supabase Postgres)
-        run: pnpm --filter @cipherstash/stack run test:integration
+        run: pnpm exec turbo run test:integration --filter @cipherstash/stack --env-mode=loose
         env:
           DATABASE_URL: ${{ steps.db.outputs.database-url }}
           PGRST_URL: ${{ steps.db.outputs.pgrest-url }}

.github/workflows/prisma-next-e2e.yml

@@ -121,7 +121,7 @@ jobs:
           done
 
       - name: Run E2E suite
-        run: pnpm --filter @cipherstash/prisma-next-example test:e2e
+        run: pnpm exec turbo run test:e2e --filter @cipherstash/prisma-next-example --env-mode=loose
 
       - name: Stop E2E Postgres container
         if: always()

.github/workflows/tests.yml

@@ -193,6 +193,21 @@ jobs:
       - name: Typecheck (stack — emitted declarations, not source)
         run: pnpm exec turbo run test:types:dist --filter @cipherstash/stack
 
+      # `stash init` writes a client file into the user's project and tells them
+      # not to hand-edit it. Nothing compiled that file, so tightening
+      # `Encryption` to require a non-empty schema set left every `stash init`
+      # emitting a project that fails its first `tsc` — with CI green (#772
+      # review). The fixtures are pinned byte-for-byte to the generator by
+      # `placeholder-client-fixture.test.ts`.
+      # Through turbo, not `pnpm run` — the fixtures import `@cipherstash/stack/v3`,
+      # so this needs that package BUILT. Invoked directly it passed only because
+      # earlier steps in this job happen to build it via their own `^build`; drop
+      # or reorder those (they read as guards for other packages, so they look
+      # independently removable) and this fails `TS2307`, which reads as "the
+      # scaffold is broken" rather than "you forgot to build" (#787 review).
+      - name: Typecheck (stash init's scaffolded client)
+        run: pnpm exec turbo run typecheck:scaffold --filter stash
+
       - name: Lint — no hardcoded package-manager runners
         run: pnpm run lint:runners
 

packages/cli/AGENTS.md

@@ -6,11 +6,24 @@ This package has **two** Vitest configs. Run the right one for the change.
 
 | Command | Config | Scope | Needs build? |
 | --- | --- | --- | --- |
-| `pnpm --filter stash test` | `vitest.config.ts` | Unit tests under `src/__tests__/**` and `src/**/__tests__/**` | No |
+| `pnpm --filter stash test` | `vitest.config.ts` | Unit tests under `src/__tests__/**` and `src/**/__tests__/**` | **Partly** — needs `@cipherstash/stack` built (see below). Turbo's `^build` supplies it in CI. |
 | `pnpm --filter stash test:e2e` | `vitest.integration.config.ts` | E2E tests under `tests/e2e/**.e2e.test.ts` driving the built `dist/bin/stash.js` through a real pty (`node-pty`) | **Yes** — run `pnpm --filter stash build` first, or use the turbo `test:e2e` task which depends on `build`. |
 
 The unit config explicitly excludes `tests/e2e/**` so the default `pnpm test`
-stays fast and self-contained.
+stays fast.
+
+It is **not** fully self-contained, despite running standalone in CI. Some `src`
+modules import workspace packages that publish `./dist` only, so an unbuilt
+workspace fails at collection with `Failed to resolve entry for package …`
+rather than at an assertion. `vitest.config.ts` aliases `@cipherstash/migrate`
+to its source to remove one such coupling; `@cipherstash/stack` remains, reached
+via `packages/migrate/src/backfill.ts` and a direct import in
+`init/lib/__tests__/introspect.test.ts`. Deleting `packages/stack/dist` fails 10
+files. Closing it needs `vitest.shared.ts`'s `stackSourceAlias`, which cannot be
+spread into this config: its `'@/'` points at `packages/stack/src` while this
+package's points at `packages/cli/src`, and a flat alias map admits only one —
+spread it after and stack's entry clobbers the CLI's, spread it before and the
+CLI's breaks stack's own source imports (#787 review).
 
 ## When to add or update an E2E test
 

packages/cli/__fixtures__/scaffold/drizzle.generated.ts

@@ -0,0 +1,67 @@
+/**
+ * CipherStash encryption client — placeholder.
+ *
+ * `stash init` wrote this file. It is intentionally NOT a real Drizzle
+ * schema. Your existing schema files (typically under `src/db/`) remain
+ * authoritative — your agent will edit those directly when you encrypt a
+ * column, then update the `Encryption({ schemas: [...] })` call below
+ * to reference the encrypted tables you declared there.
+ *
+ * Until that happens, the encryption client is initialised with a single
+ * placeholder table so that this file compiles, and `stash db push`,
+ * `stash db validate` and `stash encrypt backfill` refuse to run and point
+ * back here. (`stash encrypt cutover` / `drop` resolve against the database
+ * and never read this file.)
+ *
+ * This project uses EQL v3. Encrypted columns are concrete Postgres domains
+ * built with the `types.*` factories from `@cipherstash/stack-drizzle`.
+ * Each domain's query capabilities are FIXED by the type you pick — there is
+ * no capability config object. Choose the factory whose capabilities you need:
+ *   types.Text / types.Integer / …     storage only (encrypt/decrypt, no queries)
+ *   types.TextEq / types.IntegerEq     equality (eq, inArray)
+ *   types.IntegerOrd / types.DateOrd   equality + order/range (gt/lt/between/sort)
+ *   types.TextMatch                    free-text match only
+ *   types.TextSearch                   equality + order/range + free-text
+ *   types.Json                         encrypted-JSONB containment + selectors
+ *
+ * --- Pattern reference (copy into your real schema, do NOT use as-is) ---
+ *
+ * Encrypted twin column for an existing populated column (path 3 — lifecycle):
+ *
+ *   import { pgTable, integer, text } from 'drizzle-orm/pg-core'
+ *   import { types } from '@cipherstash/stack-drizzle'
+ *
+ *   export const users = pgTable('users', {
+ *     id: integer('id').primaryKey().generatedAlwaysAsIdentity(),
+ *     email: text('email').notNull(),                        // existing plaintext, unchanged for now
+ *     email_encrypted: types.TextSearch('email_encrypted'),  // encrypted twin, NULLABLE — never .notNull()
+ *   })
+ *
+ * Net-new encrypted column (path 1 — declare encrypted from the start):
+ *
+ *   export const orders = pgTable('orders', {
+ *     id: integer('id').primaryKey().generatedAlwaysAsIdentity(),
+ *     billing_address: types.TextEq('billing_address'),
+ *   })
+ *
+ * Once you have encrypted tables declared, harvest them and pass to Encryption():
+ *
+ *   import { extractEncryptionSchema } from '@cipherstash/stack-drizzle'
+ *   import { Encryption } from '@cipherstash/stack/v3'
+ *   import { users, orders } from './db/schema'
+ *
+ *   export const encryptionClient = await Encryption({
+ *     schemas: [extractEncryptionSchema(users), extractEncryptionSchema(orders)],
+ *   })
+ */
+import { Encryption, encryptedTable, types } from '@cipherstash/stack/v3'
+
+// REPLACE THIS. It exists only so this file compiles before you have declared
+// any encrypted tables — `Encryption` requires at least one. Swap it for your
+// real tables (see the patterns above); `stash db push`, `stash db validate`
+// and `stash encrypt backfill` refuse to run while the placeholder is still here.
+export const placeholderTable = encryptedTable('__stash_placeholder__', {
+  replace_me: types.Text('replace_me'),
+})
+
+export const encryptionClient = await Encryption({ schemas: [placeholderTable] })

packages/cli/__fixtures__/scaffold/generic.generated.ts

@@ -0,0 +1,62 @@
+/**
+ * CipherStash encryption client — placeholder.
+ *
+ * `stash init` wrote this file. It is intentionally NOT a real schema
+ * definition. Your existing schema files remain authoritative — your
+ * agent will declare encrypted columns there and update the
+ * `Encryption({ schemas: [...] })` call below to reference them.
+ *
+ * Until that happens, the encryption client is initialised with a single
+ * placeholder table so that this file compiles, and `stash db push`,
+ * `stash db validate` and `stash encrypt backfill` refuse to run and point
+ * back here. (`stash encrypt cutover` / `drop` resolve against the database
+ * and never read this file.)
+ *
+ * This project uses EQL v3. Encrypted columns are concrete Postgres domains
+ * built with the `types.*` factories from `@cipherstash/stack/eql/v3`
+ * (also re-exported from `@cipherstash/stack/v3`). Each domain's query
+ * capabilities are FIXED by the type you pick — there is no chainable
+ * capability tuner. Choose the factory whose capabilities you need:
+ *   types.Text / types.Integer / …     storage only (encrypt/decrypt, no queries)
+ *   types.TextEq / types.IntegerEq     equality
+ *   types.IntegerOrd / types.DateOrd   equality + order/range
+ *   types.TextMatch                    free-text match only
+ *   types.TextSearch                   equality + order/range + free-text
+ *   types.Json                         encrypted-JSONB containment + selectors
+ *
+ * --- Pattern reference (copy into your real schema, do NOT use as-is) ---
+ *
+ * Encrypted twin column for an existing populated column (path 3 — lifecycle):
+ *
+ *   import { encryptedTable, types } from '@cipherstash/stack/v3'
+ *
+ *   export const users = encryptedTable('users', {
+ *     email_encrypted: types.TextSearch('email_encrypted'),
+ *   })
+ *
+ * Net-new encrypted column (path 1 — declare encrypted from the start):
+ *
+ *   export const orders = encryptedTable('orders', {
+ *     billing_address: types.TextEq('billing_address'),
+ *   })
+ *
+ * Once you have encrypted tables declared, pass them to Encryption():
+ *
+ *   import { Encryption } from '@cipherstash/stack/v3'
+ *   import { users, orders } from './db/schema'
+ *
+ *   export const encryptionClient = await Encryption({
+ *     schemas: [users, orders],
+ *   })
+ */
+import { Encryption, encryptedTable, types } from '@cipherstash/stack/v3'
+
+// REPLACE THIS. It exists only so this file compiles before you have declared
+// any encrypted tables — `Encryption` requires at least one. Swap it for your
+// real tables (see the patterns above); `stash db push`, `stash db validate`
+// and `stash encrypt backfill` refuse to run while the placeholder is still here.
+export const placeholderTable = encryptedTable('__stash_placeholder__', {
+  replace_me: types.Text('replace_me'),
+})
+
+export const encryptionClient = await Encryption({ schemas: [placeholderTable] })

packages/cli/package.json

@@ -41,6 +41,7 @@
 		"postbuild": "chmod +x ./dist/bin/stash.js",
 		"dev": "tsup --watch",
 		"test": "vitest run",
+		"typecheck:scaffold": "tsc -p tsconfig.scaffold.json",
 		"test:e2e": "vitest run --config vitest.integration.config.ts",
 		"lint": "biome check ."
 	},