Skip to content

feat(pg-delta-next): orderless declarative apply, grouped export, formatting, redaction#307

Open
avallete wants to merge 59 commits into
feat/pg-delta-next-testsfrom
feat/pg-delta-next-avallete
Open

feat(pg-delta-next): orderless declarative apply, grouped export, formatting, redaction#307
avallete wants to merge 59 commits into
feat/pg-delta-next-testsfrom
feat/pg-delta-next-avallete

Conversation

@avallete

@avallete avallete commented Jun 24, 2026

Copy link
Copy Markdown
Member

What kind of change does this PR introduce?

Feature + fixes. This is the GREEN half of the TDD split: the source for the
next pg-delta engine. It is stacked on the tests PR #315 — its base is the
feat/pg-delta-next-tests branch, so the diff here is source-only and turns the
tests in #315 green.

Merge #315 first, then this PR.

What's in here

  • The next-engine source: extraction (publications, foreign, sensitive options,
    scope, dependencies), planner (action-graph/emitter, policy/schema/table/type
    rules), frontends (sql-order shadow reorder, grouped export, SQL formatter),
    CLI commands, policy/supabase integration, and the pg-topo total-order change.
  • Per-behavior changesets.

CI fixes (were failing on the combined branch)

  • Type check (Cannot find module '@supabase/pg-topo'): sql-order.ts
    imports the optional @supabase/pg-topo peer. tsc resolves it via the
    package's types field (dist/*.d.ts), which is gitignored and not produced
    by bun install. Added a step to build @supabase/pg-topo before the type
    check; runtime continues to use the bun source export condition.
  • Corpus PG14 (drop-publication-fk-chain-partial-membership,
    syntax error at or near "TABLE"): the inlined CREATE PUBLICATION … FOR
    clause repeated the TABLE keyword per relation (FOR TABLE a, TABLE b),
    valid only on PG15+. Now grouped under a single TABLE (and a single
    TABLES IN SCHEMA), valid on every supported version. Full corpus
    554 pass / 0 fail on postgres:14-alpine and postgres:17-alpine.

Restored docs

Restores docs/roadmap/** (6 files) that an earlier PR-shrinking commit removed.
The dogfooding harness and run artifacts remain removed.

@changeset-bot

changeset-bot Bot commented Jun 24, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: da8ce04

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@supabase/pg-delta-next Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

avallete added a commit that referenced this pull request Jun 26, 2026
… docs

Shrinks PR #307 to the minimal pg-delta-next feature set by removing the
dogfooding / engine-comparison scaffolding and planning docs that were not
part of the shipped engine:

- docs/dogfooding/** (run artifacts, comparison rubric, HTML reports)
- docs/roadmap/**, docs/old-vs-north-star.html, scripts/md-to-html.ts
- pg-delta-next dogfood runners (run-dogfood-suite, run-bookmark-dogfood,
  generate-dogfood-report) + their report-/paths-/bookmark-fixture libs
- the old-vs-new comparison harness (compare-engines, compare-core,
  tests/old-engine.ts, tests/differential.test.ts, tests/compare-applycheck.test.ts)

Kept scripts/lib/bootstrap-dbdev-fixture.ts (needed by dbdev-roundtrip.test.ts)
and updated its header comment. Trimmed the README Dogfooding section.

Parity with the old engine is still tracked via porting-ledger.json +
audit-porting-ledger.ts (porting-ledger.test.ts stays green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
avallete and others added 3 commits June 26, 2026 16:35
…matting, redaction

The pg-delta-next engine source: extraction (publications, foreign, sensitive
options, scope, dependencies), planner (action-graph/emitter, policy/schema/
table/type rules), frontends (sql-order shadow reorder, grouped export, SQL
formatter), CLI commands, policy/supabase integration, and the pg-topo
total-order change. Includes the changesets describing each behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sql-order.ts imports the optional @supabase/pg-topo peer. tsc resolves it via
the package's `types` field (dist/*.d.ts), which is gitignored and not
produced by `bun install`, so the type-check job failed with
"Cannot find module '@supabase/pg-topo'". Build the package first so the
declaration files exist; runtime continues to use the `bun` source condition.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…PG14)

The inlined FOR-clause repeated the keyword per relation
(`FOR TABLE a, TABLE b`), which is only valid grammar on PG15+. On PG14 it is
a syntax error, so the corpus scenario
dependencies-cycles--drop-publication-fk-chain-partial-membership failed to
apply in the teardown direction. Group all table relations under a single
`TABLE` and all schemas under a single `TABLES IN SCHEMA`; the collapsed form
is valid on every supported version.

RED (before): apply failed at action 9: syntax error at or near "TABLE"
  CREATE PUBLICATION "supabase_realtime" FOR TABLE "public"."public_offering_events", TABLE "public"."trades" ...
GREEN (after): full corpus 554 pass / 0 fail on postgres:14-alpine and postgres:17-alpine.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@avallete avallete force-pushed the feat/pg-delta-next-avallete branch from d5785e2 to 73ee0b8 Compare June 26, 2026 14:49
@avallete avallete changed the base branch from feat/pg-delta-next to feat/pg-delta-next-tests June 26, 2026 14:49
@avallete avallete changed the title chore: improve pg delta next avallete feat(pg-delta-next): orderless declarative apply, grouped export, formatting, redaction Jun 26, 2026
Comment thread packages/pg-delta-next/PORTING-GAPS.md Outdated
@avallete

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 73ee0b8e39

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/pg-delta-next/src/cli/commands/schema.ts Outdated
Comment thread packages/pg-delta-next/src/cli/commands/schema.ts
Comment thread packages/pg-delta-next/src/frontends/sql-format/index.ts
Comment thread packages/pg-delta-next/src/plan/internal.ts
Comment thread packages/pg-delta-next/src/plan/rules/types.ts
Comment thread .github/workflows/pg-delta-next.yml
@avallete avallete marked this pull request as ready for review June 29, 2026 08:06
Removed the porting-ledger audit step from the workflow.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 73ee0b8e39

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/pg-delta-next/src/cli/commands/schema.ts Outdated
Comment thread packages/pg-delta-next/src/extract/sensitive-options.ts
Comment thread packages/pg-delta-next/src/cli/commands/schema.ts Outdated
avallete and others added 3 commits June 29, 2026 17:16
RED coverage for the PR #307 review findings, each confirmed failing
against the pre-fix engine for the right reason:

- password_required redacted to a placeholder (sensitive-options)
- BEGIN ATOMIC / CASE…END bodies shredded by the format splitter
- domain NOT VALID constraint inlined → "syntax error at or near VALID"
  (corpus: domain-operations--not-valid-constraint)
- profile export throws "missing requirement: … consumes schema:extensions"
- schema apply silently drops a pg-topo parse-error file / reorders across
  SET search_path|ROLE barriers (CLI reorder safety)
- schema apply re-redacts real credentials → placeholder on target

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turns the regressions in the preceding test commit green:

- sensitive-options: allowlist non-secret `password_required` (P2)
- sql-format: BEGIN ATOMIC/CASE…END-aware statement splitter so
  `--format-options` keeps a routine body as one statement (P2)
- plan/rules/types: don't inline NOT VALID domain constraints into
  CREATE DOMAIN; emit standalone ALTER DOMAIN … ADD CONSTRAINT (P2)
- export: forward profile assumedSchemas/assumedRoles into the internal
  plan; plan() now accepts them directly (P1)
- cli schema apply: fall back to raw, file-granular loading (+warning)
  on a pg-topo parse/discovery error or when files contain session
  settings (SET search_path / SET ROLE) instead of silently dropping or
  reordering them (P1)
- cli schema apply: add --unsafe-show-secrets to round-trip real
  credentials through both shadow and target extracts (P2)

#5 (owner-ACL elision of owner-revoked defaults) is deferred as a
follow-up — it needs version-aware owner full-privilege detection.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…S.md

- pg-delta-next.yml: add packages/pg-topo/** to the pull_request and push
  path filters, since the workflow builds and type/integration-checks
  against @supabase/pg-topo (P2).
- Remove packages/pg-delta-next/PORTING-GAPS.md (porting complete; the
  ledger gate it documented was already dropped, and nothing references it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb4c83045d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/pg-delta-next/src/frontends/sql-format/index.ts
Comment thread packages/pg-delta-next/src/frontends/sql-format/formatters.ts
Comment thread packages/pg-delta-next/src/frontends/sql-format/formatters.ts Outdated
Comment thread packages/pg-delta-next/scripts/verified-coverage.json Outdated
avallete and others added 3 commits June 30, 2026 10:12
RED for the second Codex review pass on PR #307 (--format-options path):
- DO ALSO (…; …) rewrite-rule body split on inner semicolons
- index INCLUDE (…) closing paren cut before a WHERE/WITH/TABLESPACE clause
- index NULLS NOT DISTINCT dropped before such a clause

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tting

Addresses the second Codex review pass on PR #307:
- splitSqlStatements suppresses `;` splitting at parenthesis depth > 0, so a
  multi-command rewrite-rule body `DO ALSO ( INSERT …; UPDATE … )` stays one
  statement (in addition to the BEGIN ATOMIC/CASE block tracking).
- formatCreateIndex restores the trimmed leading-whitespace offset so an
  INCLUDE (…) list keeps its closing paren before a WHERE/WITH/TABLESPACE
  clause, and keeps the modifier text (e.g. NULLS NOT DISTINCT) between the
  list and the first recognized clause on the header line instead of dropping it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per review, remove scripts/verified-coverage.json (a porting-era artifact).
Its only consumer was scripts/seed-porting-ledger.ts — a one-shot seeder that
built tests/porting-ledger.json from it — so remove that script and its
`seed:porting` package.json entry too. The audit side (`audit:porting` /
audit-porting-ledger.ts) and the ledger + its test are independent of the
removed file and stay intact (`audit:porting` still reports the ledger in sync).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

const report = await apply(thePlan, tgt.pool, {
...ctx.applyOptions, // baseline + handler-aware re-extract (from the profile)
fingerprintGate: !force,

P2 Badge Re-extract unredacted state for unsafe schema apply

When schema apply --unsafe-show-secrets plans against a target that already contains FDW/server/user-mapping credentials or subscription conninfo, the source fingerprint is built from the unredacted targetResult, but the immediate apply still uses ctx.applyOptions.reextract, whose default extractor redacts secrets. Fresh evidence beyond the prior shadow/target extraction fix is that this apply() call does not wrap reextract with the same { redactSecrets }, so the fingerprint gate can abort unless --force is used even though the command opted into unredacted round-tripping.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/pg-delta-next/src/frontends/sql-format/formatters.ts
Comment thread packages/pg-delta-next/src/cli/commands/schema.ts Outdated
Comment thread packages/pg-delta-next/src/policy/policy.ts
Comment thread packages/pg-delta-next/src/frontends/sql-format/formatters.ts Outdated
avallete and others added 2 commits June 30, 2026 11:49
… ACL

RED for review finding #5 (owner-ACL elision). When an object's owner revokes
one of their own create-time default privileges, the default-ACL compaction
elided the whole REVOKE/GRANT group, so the applied object kept PostgreSQL's
full owner default and the plan never converged.

- corpus default-privileges-edge-case--owner-revoke-own-default: `CREATE TABLE;
  REVOKE UPDATE ON t FROM CURRENT_USER` planned from empty. RED proof:
    plan: [CREATE TABLE]   (owner REVOKE/GRANT elided)
    drift: acl…privileges […,UPDATE] -> […]   (UPDATE wrongly came back)
- internal.test.ts: elideDefaultAclCreates keeps the owner group when the
  owner's grant is a strict subset of its create-time default.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turns the preceding regression green. The owner's create-time default privilege
set is captured from acldefault() at extract time (version-correct — PG17 added
MAINTAIN) and carried on the owner ACL fact as `ownerDefault`. The two default-
ACL compaction passes now use it:

- elideDefaultAclCreates drops the owner REVOKE/GRANT group only when the desired
  owner privileges EXACTLY equal that default (was: always, regardless of the
  privilege list).
- elideCoCreateRevokeBeforeGrant never strips the leading REVOKE from an owner
  group — for the owner the REVOKE is load-bearing (the owner starts with the
  full default), and any owner group reaching that pass is a strict subset.

`ownerDefault` is owner-only and identical on both diff sides for a given PG
version, so it adds no diff/drift signal. Full corpus: 558 pass / 0 fail on
postgres:17-alpine (sharded).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

const report = await apply(thePlan, tgt.pool, {
...ctx.applyOptions, // baseline + handler-aware re-extract (from the profile)
fingerprintGate: !force,

P2 Badge Preserve unsafe-secret fingerprints during apply

When schema apply --unsafe-show-secrets runs against a target that already has any unredacted FDW/server/user-mapping option or subscription conninfo, the plan source fingerprint is computed from the unredacted targetResult.factBase, but this call still uses ctx.applyOptions.reextract, whose profile re-extractor defaults back to redaction. The fingerprint gate then compares a redacted re-extract to the unredacted plan fingerprint and rejects the one-shot apply unless the user disables the gate with --force.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/pg-delta-next/src/frontends/sql-format/formatters.ts Outdated
Comment thread packages/pg-delta-next/src/frontends/sql-format/formatters.ts Outdated
Comment thread packages/pg-delta-next/src/frontends/sql-format/format-utils.ts
avallete and others added 3 commits June 30, 2026 12:53
…ndings

RED for the third Codex review batch on PR #307:
- formatter drops the clause after a QUOTED object name (trigger event/table,
  server FOREIGN DATA WRAPPER, subscription CONNECTION; + FDW/language)
- formatter splits a keyword-like qualified type name (RETURNS public.cost,
  column public.generated)
- schema apply fails when the optional @supabase/pg-topo peer is absent
- schema apply --unsafe-show-secrets trips the fingerprint gate against a target
  that already holds unredacted secrets

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ingerprint

Turns the preceding regressions green.

Formatter (--format-options):
- add identifierEnd() and use it in the trigger/server/subscription/FDW/language
  formatters to locate a (possibly quoted) object name from the raw statement,
  instead of positional token indexing that scanTokens shifts past a quoted name
  onto the next clause keyword (dropping that clause).
- in the function clause scan and parseDefinitionItem boundary scan, skip a
  keyword that is the tail of a qualified name (preceded by `.`) so a type like
  public.cost / public.generated is not split.

schema apply:
- catch ReorderUnavailableError (optional @supabase/pg-topo peer absent) and fall
  back to raw, file-granular loading with a warning.
- pass redactSecrets into the apply re-extract so the fingerprint gate compares
  like-for-like under --unsafe-show-secrets.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per review of #3497360394 (decided not to fix): the apply fingerprint gate folds
the whole resolved view, including referenceOnly assumed-schema facts, so a
platform change to an unmanaged assumed-schema object between plan and apply
trips the gate and asks for a re-plan. This is intentional — plan and apply must
run against the same baseline for the plan to be provably applicable; if the
baseline shifted, regenerating the plan is the correct response. Documented at
the gate (apply.ts) and on FactBase.rootHash so the trade-off is explicit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
avallete and others added 2 commits July 1, 2026 12:35
…-load warning, placeholder collision

RED evidence (production stashed):

protect.test.ts:
  (fail) placeholder collision safety > does not clobber an identifier equal to the placeholder token
    Received: `CREATE VIEW "AS SELECT 1" AS SELECT 1`  (view name clobbered by restore)

export-manifest.test.ts:
  error: Cannot find module './export-manifest.ts'  (helper did not exist)

cli.test.ts:
  (fail) --no-reorder still warns about ADP raw-load ordering
    no ADP NOTE on the --no-reorder raw path
  (fail) schema export records its redaction mode in a manifest / apply honors it
    .pgdelta-export.json not written; apply re-extracted redacted

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ths; collision-proof placeholder

- schema export writes .pgdelta-export.json (redaction mode); schema apply --dir
  re-extracts with it, so an --unsafe-show-secrets export round-trips real
  credentials without re-passing the flag. Flag is the manifest-less fallback;
  the .json sidecar is ignored by the .sql-only loader/pruner (#3505088638).
- The ADP raw-load caveat NOTE now fires on every raw path (--no-reorder, missing
  pg-topo peer), not only the diagnostics-disabled branch (#3505088640).
- The formatter placeholder uses a sentinel prefix guaranteed absent from the
  input, so SQL literally containing the token isn't clobbered on restore (#3505088644).

GREEN: 570 unit pass; cli + export integration 43 pass; types + lint clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

name: full.slice(dir.length + 1), // relative path from dir

P2 Badge Normalize --dir before deriving SQL file names

When --dir is passed with a trailing slash or a non-normalized segment, join(current, entry) normalizes full but this slice still uses the raw dir.length + 1, so the first character of every relative path is dropped. The corrupted name is then used by loadSqlFiles for lexicographic ordering, so raw-load paths (--no-reorder, missing pg-topo, or reorder barriers like ADP/session settings) can run files in the wrong order; for example 01_schema.sql becomes 1_schema.sql while 10_table.sql becomes 0_table.sql, causing the table file to be attempted before the schema. Compute the relative name from normalized paths (e.g. relative(resolve(dir), full)) instead of slicing the user-supplied string.


const result = await extractShadow(shadow, { source: "sqlFiles" });

P2 Badge Filter preexisting shared roles from shadow extracts

In the default databaseScratch mode, a shadow database can live on a cluster that already has non-pg_ roles unrelated to the declarative files; the before/after leak check only verifies those roles did not change, but this extraction still includes them in the desired fact base. If the target cluster lacks one of those preexisting shadow roles, schema apply will plan to create it even though no input .sql file declared it; subtract the pre-load shared-role snapshot or otherwise reject a non-isolated shadow cluster with extra roles before returning the extracted state.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/pg-delta-next/src/cli/commands/schema.ts Outdated
Comment thread packages/pg-delta-next/src/cli/commands/schema.ts Outdated
avallete and others added 2 commits July 1, 2026 13:00
…ty-dir guards and --dir normalization

RED evidence (schema.ts + export-manifest.ts stashed):

collect-sql-files.test.ts:
  SyntaxError: Export named 'collectSqlFiles' not found  (not exported)
  (with trailing slash the raw slice would yield "1_schema.sql" etc.)

export-manifest.test.ts:
  SyntaxError: Export named 'readExportManifest' not found

cli.test.ts:
  (fail) refuses an empty --dir  — apply proceeded (exit 0), planning drops
  (fail) refuses to apply a profiled export under a contradicting --profile  — exit 0
  (fail) schema export records its redaction mode in a manifest  — manifest.profile undefined

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mpty --dir; normalize --dir names

- schema export records the projection profile in .pgdelta-export.json; schema
  apply --dir defaults to it and rejects a contradicting --profile up front
  (effectiveProfileId), so a supabase export isn't applied as raw and dropping
  platform state (#3505238081).
- schema apply aborts (exit 2) on an empty --dir instead of planning to drop every
  managed object on the target (#3505238083).
- collectSqlFiles derives names via relative(resolve(dir), full), so a trailing
  slash no longer drops the first char of every name and corrupts raw-load order.

Deferred (review-body P2, no thread): filtering pre-existing shared roles from the
databaseScratch shadow extract — pre-existing behavior, risky in the diff path
(role/ownership/membership edges), lower severity, sidestepped by --isolated-shadow.

GREEN: 572 unit pass; cli + export integration 43 pass; types + lint clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@avallete

avallete commented Jul 1, 2026

Copy link
Copy Markdown
Member Author

On the two review-body P2s in the latest Codex summary (no inline threads):

  • Normalize --dir before deriving SQL file names (schema.ts:102): fixed in a1ea93d. collectSqlFiles now uses relative(resolve(dir), full) instead of slicing the raw --dir string, so a trailing slash no longer drops the first character of every name (which corrupted raw-load lexicographic order). Covered by collect-sql-files.test.ts (RED→GREEN, tested with and without a trailing slash).

  • Filter preexisting shared roles from shadow extracts (load-sql-files.ts:610): deferred, with rationale. This is pre-existing behavior (not introduced by this PR): in databaseScratch mode the shadow extract includes any non-pg_ roles already on the cluster, so if the target lacks one, apply plans a spurious CREATE ROLE. The safe fix is genuinely risky — subtracting role facts collides with owner/membership edges in the diff path (the connection role owns the shadow objects), and outright rejecting a shared cluster with extra roles breaks common CI setups. It is also lower severity (an extra role, not a destructive drop) and is fully sidestepped by the --isolated-shadow mode this PR added (and the docs recommend). Happy to take it on as a focused follow-up if you want it, but it felt disproportionate to fold into this PR.

🤖 Addressed by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a1ea93d032

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/pg-delta-next/src/cli/commands/schema.ts Outdated
Comment thread packages/pg-delta-next/src/cli/commands/schema.ts Outdated
Comment thread packages/pg-delta-next/src/frontends/sql-format/formatters.ts
avallete and others added 2 commits July 1, 2026 13:41
…pty-export root, CREATE SERVER FDW name

RED evidence (schema.ts + formatters.ts stashed):

collect-sql-files.test.ts:
  SyntaxError: Export named 'writeExportFiles' not found (helper absent; zero-file
  export would ENOENT on the manifest write)

format-quoted-names.test.ts:
  (fail) foreign server keeps an unquoted keyword-like FDW name (e.g. options)
    Received: "... FOREIGN DATA WRAPPER\n  options\n  OPTIONS (host 'h')" — name
    orphaned as its own OPTIONS clause

cli.test.ts:
  (fail) refuses a directory whose only .sql is comment-only — apply proceeded (exit 0)
  (fail) refuses an empty --dir — message changed to "no executable SQL"

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ot for empty export; FDW name in CREATE SERVER scan

- schema apply refuses a --dir with no executable SQL (scanTokens skips
  comments/strings), so a placeholder/comment-only file no longer yields an empty
  shadow that plans to drop every managed object (#3505497725).
- schema export writes files+manifest via writeExportFiles, which creates outRoot
  up front so a zero-file (no managed objects) export doesn't ENOENT (#3505497730).
- formatCreateServer skips the FDW name after FOREIGN DATA WRAPPER, so an unquoted
  non-reserved name like `options` isn't misread as an OPTIONS clause (#3505497733).

GREEN: 575 unit pass; cli + export + format integration 38 pass; types + lint clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

{ kind: ["acl", "comment", "securityLabel"] },
{
any: [
{ target: { schema: [...SUPABASE_SYSTEM_SCHEMAS] } },

P2 Badge Keep metadata on included Supabase triggers

This blanket satellite exclusion also catches metadata for objects that an earlier include rule deliberately keeps, such as a user trigger on auth.users whose function is in public. A COMMENT ON TRIGGER fact has kind comment and target schema auth, so it is pruned while the trigger remains managed; export/plan then silently drops the user's trigger comment. Exclude satellites only when their target is actually excluded/reference-only, or add a matching include for satellites of the included trigger.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// (scanTokens skips comments/strings). A wrong/empty --dir must be a loud error,
// not a silent destructive plan (review P1).
const files = collectSqlFiles(dir);
const hasExecutableSql = files.some((f) => scanTokens(f.sql).length > 0);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject schema-neutral SQL-only apply dirs

This guard still treats any tokenized statement as a non-empty desired schema, so a wrong directory containing only schema-neutral SQL such as SELECT 1; or SET statement_timeout = 0; passes the check, loads an empty shadow, and then plans from the live target to an empty desired state. Because the fingerprint source is the same live target, the gate can pass and schema apply can drop every managed object; validate that the loaded shadow produced at least one managed fact or require an explicit empty-desired mode.

Useful? React with 👍 / 👎.

Comment on lines +53 to +54
} catch {
return undefined;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fail on malformed export manifests

When .pgdelta-export.json exists but is malformed or unreadable, this catch returns undefined, so schema apply --dir silently falls back to the CLI/default profile and redaction mode. For a --profile supabase or --unsafe-show-secrets export with a truncated/merged manifest, that bypasses the safety metadata and can reintroduce the raw-profile destructive behavior or redact real credentials to placeholders; distinguish a missing manifest from an invalid one and abort on the latter.

Useful? React with 👍 / 👎.

}
return i; // unterminated — caller still gets a sane bound
}
while (i < statement.length && isWordChar(statement[i]!)) i += 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include dollar signs in identifier scans

PostgreSQL allows dollar signs in unquoted identifiers after the first character, but this scanner stops the name before $. With --format-options, valid statements like CREATE SERVER srv$1 FOREIGN DATA WRAPPER ... or CREATE MATERIALIZED VIEW public.v$1 AS ... are sliced with the $1 portion before the first clause and sliceClauses() drops it, so the formatted SQL targets srv/public.v instead of srv$1/public.v$1; include $ when scanning unquoted identifiers without treating it as a dollar quote inside identifiers.

Useful? React with 👍 / 👎.

const path = subject === undefined ? "cluster/misc.sql" : pathFor(subject);
const entry = files.get(path) ?? { firstAt: position, statements: [] };
entry.statements.push(`${action.sql};`);
entry.statements.push(action.sql);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Split by-object files across dependency gaps

Because this unconditionally appends every action for the same object path into one file, the default by-object export can create file-level cycles that the raw loader cannot break. For example, a valid mutual-FK state with CREATE a, CREATE b REFERENCES a, then ALTER a ADD CONSTRAINT ... REFERENCES b is exported as a.sql containing CREATE a plus the later constraint and b.sql containing CREATE b; loadSqlFiles/schema apply --no-reorder wraps each file atomically, so a.sql rolls back waiting for b and b.sql rolls back waiting for a. Split non-consecutive runs, as ordered layout does, or otherwise avoid grouping plan steps across dependency gaps.

Useful? React with 👍 / 👎.

avallete and others added 2 commits July 1, 2026 16:31
… REVOKE … FROM PUBLIC

RED evidence (before the fix), corpus a->b:
  proof failed
  drift:  add defaultPrivilege:test."".f.test
  plan:   0: ALTER DEFAULT PRIVILEGES FOR ROLE "test" GRANT EXECUTE ON FUNCTIONS TO "test"
  (the required REVOKE EXECUTE … FROM PUBLIC was never emitted, so the clone kept
   PUBLIC's built-in default and never converged)

Adds corpus scenarios (functions EXECUTE, types USAGE; both directions) and a
renderer unit test for the revoked-default marker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…uilt-in default

pg_default_acl materializes the whole effective default, so a revoked built-in
default (ALTER DEFAULT PRIVILEGES REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC / REVOKE
USAGE ON TYPES FROM PUBLIC) showed up only as an absent grantee, and the GRANT-only
renderer could not reproduce it — the hardening was silently lost on apply/export.

Extraction (roles.ts) now emits a fact only when a grantee DEVIATES from its
built-in default (derived from acldefault(), version/kind-robust): a grantee at
default → no fact; a revoked default → an empty marker carrying _revokedDefault
(non-semantic `_` metadata, excluded from hash/diff). Rendering (helpers.ts):
marker → create REVOKE ALL, drop GRANT _revokedDefault; positive grant → create
GRANT, drop REVOKE ALL. This also removes the spurious owner-default facts that
previously caused a b->a REVOKE of the owner's built-in default.

GREEN: full corpus pg17 568/568 and pg15 568/568; 577 unit; types + lint clean.
Known limitation documented in the changeset (partial reduction of a grantee that
has a built-in default — exotic; PUBLIC single-priv defaults are always exact).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

SELECT count(*)::int AS n FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname NOT IN ('pg_catalog', 'information_schema')
AND n.nspname NOT LIKE 'pg\\_%'`);

P2 Badge Reject shadows with schema-only leftovers

With a reused shadow that contains only an empty user schema, function, type, or other managed object not represented in pg_class, this emptiness check still passes because it counts only relation rows. The stale object is then included in the extracted desired state even if no input file declares it, so schema apply can create or preserve state from a previous run; verify emptiness via extraction or include all managed catalogs while excluding built-ins like public.



P2 Badge Preserve session-setting files as load barriers

When a file containing SET search_path, SET ROLE, or SET SESSION AUTHORIZATION initially fails, pushing it into next lets later files run in the same round after that file's transaction rolled back the setting. For lexicographic files like 00_set_and_create.sql, 01_schema.sql, and 02_unqualified_create.sql, the third file can create objects under the wrong schema/role before the setting file succeeds in round 2; session-setting files need a prefix-preserving barrier or hard failure instead of ordinary defer-and-retry.


FROM pg_user_mapping u
JOIN pg_foreign_server s ON s.oid = u.umserver
LEFT JOIN pg_roles r ON r.oid = u.umuser
ORDER BY s.srvname, 2`)) {

P2 Badge Filter mappings whose server was not extracted

This query extracts every user mapping even when its foreign server was omitted by the server extractor's extension-member filtering. If an extension/platform supplies a server that is not extracted, a user mapping for it is pushed with parent: { kind: "server", ... }, and buildFactBase aborts with a missing parent before policy can filter it; either retain the server as reference-only or skip mappings whose server is outside the extracted view.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +101 to +105
def AS (
SELECT x.grantee AS grantee_oid,
array_agg(x.privilege_type ORDER BY x.privilege_type) AS privileges
FROM aclexplode(acldefault(${defaclCode}, d.defaclrole)) x
GROUP BY 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Treat schema-scoped default ACL rows as additive

For IN SCHEMA default-privilege rows, PostgreSQL stores only privileges to add to the global/hard-wired defaults, not a full replacement ACL (docs: https://www.postgresql.org/docs/current/catalog-pg-default-acl.html). Comparing every row to acldefault() makes a normal ALTER DEFAULT PRIVILEGES IN SCHEMA app GRANT SELECT ON TABLES TO reader look like it also revoked the owner's built-in table privileges, because the per-schema defaclacl has no owner row; the UNION below then synthesizes an empty owner marker and export/apply emits REVOKE ALL ... FROM owner for future objects. Use an empty baseline for non-global entries and reserve acldefault() comparison for global rows.

Useful? React with 👍 / 👎.

Comment on lines +517 to +519
return {
sql: `${defaultPrivPrefix(id)} REVOKE ALL ON ${objtype} FROM ${grantee}`,
consumes,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve built-in grants when dropping grant-option deviations

When the removed default-privilege fact only added grant option to a built-in grant, e.g. ALTER DEFAULT PRIVILEGES GRANT EXECUTE ON FUNCTIONS TO PUBLIC WITH GRANT OPTION, the desired default state now has no fact because plain PUBLIC EXECUTE equals the built-in default. Dropping the source fact with REVOKE ALL removes the base EXECUTE privilege too, so future functions end up with no PUBLIC execute instead of just losing the grant option; this path needs to revoke only the grant option or otherwise restore the built-in grant after removing the deviation.

Useful? React with 👍 / 👎.

Comment on lines +231 to +234
{ re: /^set\s+(?:session\s+|local\s+)?role\b/i, label: "SET ROLE" },
{
re: /^set\s+(?:session\s+|local\s+)?session\s+authorization\b/i,
label: "SET SESSION AUTHORIZATION",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reset session settings after raw shadow loads

When a declarative file contains session-level SET ROLE, SET SESSION AUTHORIZATION, or SET search_path, the raw loader runs all files on one pooled client and only resets check_function_bodies before releasing it. A successful SET ROLE persists after COMMIT, so body validation and the post-load extraction can run under the wrong role/search path (or leak that state to the next pool user), which can hide catalog details such as foreign user-mapping options or produce permission failures; reset these session settings before validation/release or isolate them from extraction.

Useful? React with 👍 / 👎.

avallete and others added 2 commits July 2, 2026 13:55
…bient roles)

RED evidence (view.ts + schema.ts stashed):

scope.test.ts:
  Export named 'projectManagementScope' not found (helper absent)

cli.test.ts (cross-cluster shadow/target):
  (fail) --scope database > does not create shadow-only nor drop target-only cluster roles
    expect(has(targetRole)).toBe(true) — Received: false
    (the target-only role was DESTRUCTIVELY dropped; a shadow-only role would be created)

Also adapts the --isolated-shadow role test to the new decoupled semantics
(managing roles now requires --scope cluster, not --isolated-shadow alone).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mbient roles in database scope

Adds projectManagementScope (policy/view.ts): database scope removes role/membership
facts and their owner edges (reusing excludeFactsAndDescendants); cluster scope is
identity. cmdSchemaApply gains --scope (default database), validates that cluster
scope requires --isolated-shadow, captures the target's role names as assumedRoles
(so grants resolve / a missing grantee fails at plan time), and applies the
projection to both diff sides AND the fingerprint re-extract so plan == prove == run.

Fixes the shared-cluster shadow-roles gap: a cross-cluster apply no longer plans a
spurious CREATE ROLE (shadow-only) or a destructive DROP ROLE (target-only).
--isolated-shadow now controls only shadow placement; role management is --scope
cluster.

GREEN: 580 unit; full cli integration 36 pass; types + lint clean. Corpus
unaffected (projection is CLI-only; plan()/engine unchanged).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4b0144050d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +626 to +627
loadResult = await loadSqlFiles(loadInput, shadow.pool, {
extract: (p, o) => ctx.extract(p, { ...o, redactSecrets }),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Seed ambient target roles before loading the shadow

In the default --scope database path with a separate shadow cluster, declarative SQL that grants to or assigns ownership to a role that exists on the target but not on the shadow still fails during loadSqlFiles (e.g. GRANT SELECT ON app.t TO app_user raises “role does not exist”). The target roles are only collected later in assumedTargetRoles, so the planner can treat them as ambient but the shadow never gets far enough to extract the desired ACL/owner state; extract/seed ambient target roles before loading, or otherwise make the shadow able to resolve target-local role names.

Useful? React with 👍 / 👎.

Comment on lines +95 to +99
if (fact.id.kind === "role" || fact.id.kind === "membership") {
roots.add(encodeId(fact.id));
}
}
return excludeFactsAndDescendants(fb, roots);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Drop shadow-owner default ACLs in database scope

When schema apply runs in the default database scope against a shadow connected as a different role than the target applier, a plain CREATE TABLE in the SQL files extracts an implicit owner ACL for the shadow login. This projection removes the role facts/owner edges but leaves that ACL fact behind, so planning can try to REVOKE/GRANT privileges for the shadow-only grantee (or fail the missing-role guard) even though ownership is explicitly unmanaged in this scope; database scope needs to discard or normalize implicit owner ACLs that only come from the shadow owner while preserving authored grants to real target roles.

Useful? React with 👍 / 👎.

Comment on lines +39 to +40
const { factBase, pgVersion, diagnostics } = await extract(src.pool, {
redactSecrets,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Extract snapshots with the selected profile

When a plan is produced with an integration profile such as --profile supabase, the desired snapshot used by prove is still produced here with the raw core extractor, so the snapshot lacks profile handler metadata such as managedBy edges that the profiled plan/proof path expects. In that scenario prove --profile supabase compares a handler-projected clone against a raw desired snapshot and can report false drift for operational extension objects; snapshot needs to accept/stamp the profile or prove must reject snapshots that were not captured with the plan's profile.

Useful? React with 👍 / 👎.

avallete and others added 2 commits July 2, 2026 14:20
…ster-DDL guard

RED evidence (load-sql-files.ts + export-manifest.ts + schema.ts stashed):

load-sql-files.test.ts:
  Export named 'findClusterDdlStatements' not found (scanner/stripper absent)

export-manifest.test.ts:
  (fail) round-trips the redaction mode, profile, and scope — scope not persisted

cli.test.ts: export --scope omits/keeps cluster/roles.sql + manifest scope;
apply rejects contradicting --scope; apply --scope database rejects cluster DDL
(with --skip-cluster-ddl ledger). New flags were unknown before the fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…scope, part 2)

- schema export --scope database|cluster: database scope projects roles/memberships
  out of the exported view (no cluster/roles.sql; reloads on any cluster) and stamps
  scope in the manifest; schema apply defaults to it and rejects a contradicting flag.
- schema apply --scope database refuses cluster-global DDL in files up front
  (findClusterDdlStatements) with scope-framed escapes; --skip-cluster-ddl strips
  them (stripClusterDdl, block-aware) and logs each skip.

GREEN: 584 unit; full cli integration 40 pass; types + lint clean. Engine/corpus
untouched (frontend + CLI only).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ab860ba7b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +785 to +786
const sourceFb = projectManagementScope(targetResult.factBase, scope);
const desiredFb = projectManagementScope(loadResult.factBase, scope);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Apply profile filters before dropping owner edges

In the default --scope database path, these lines strip role facts and owner edges before plan() gets to run the selected profile's resolveView. That breaks owner-based profile filters such as the Supabase system-role rule in policy/supabase.ts, so schema apply --profile supabase can treat target-only platform objects outside the system schemas but owned by supabase_admin/other system roles as managed drift and plan drops for them. Apply the profile/policy projection while the owner edges are still present, then project database scope afterward.

Useful? React with 👍 / 👎.

roots.add(encodeId(fact.id));
}
}
return excludeFactsAndDescendants(fb, roots);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve reference-only facts when projecting scope

Fresh evidence after the earlier reference-only export fix: this database-scope projection rebuilds the fact base via excludeFactsAndDescendants, which drops fb.referenceOnly for all surviving facts. During schema export --profile supabase with the default database scope, any surviving user role (for example postgres) makes this path run, so assumed-platform parents like auth.users lose their reference-only marker before exportSqlFiles can seed them into the pristine baseline, and the export can emit Supabase platform objects as managed CREATEs; carry the surviving referenceOnly set through the scope projection.

Useful? React with 👍 / 👎.

Comment on lines +298 to +300
{ re: /^\s*create\s+(role|user|group)\b/i, label: "CREATE ROLE" },
{ re: /^\s*alter\s+(role|user|group)\b/i, label: "ALTER ROLE" },
{ re: /^\s*drop\s+(role|user|group)\b/i, label: "DROP ROLE" },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not classify USER MAPPING as role DDL

These role aliases also match valid CREATE/ALTER/DROP USER MAPPING statements because USER is followed by a word boundary before MAPPING. In default database scope, a directory/export that includes an FDW user mapping is therefore rejected as CREATE ROLE (or stripped by --skip-cluster-ddl), even though the mapping is database-local state that should be loaded against ambient roles; exclude USER MAPPING from these role-DDL patterns.

Useful? React with 👍 / 👎.

avallete and others added 2 commits July 2, 2026 14:39
…quick mode)

RED (before the fix): the shadow module did not exist (withDatabaseName import
fails) and --shadow was required, so `schema apply --dir X --target Y` (no
--shadow) exited 2 instead of provisioning a co-located shadow.

Adds withDatabaseName unit tests and a co-located quick-mode integration test
(provisions a shadow on the target cluster, applies, and confirms the throwaway
database is dropped).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…omitted

Adds src/cli/shadow.ts (provisionCoLocatedShadow): creates a pgdelta_shadow_<unique>
database on the target's cluster from template0, returns its URL + a best-effort
DROP cleanup (WITH FORCE), and probes CREATEDB up front with a friendly error.
cmdSchemaApply makes --shadow optional: without it, a co-located shadow is
provisioned (database scope only; --scope cluster still needs an explicit --shadow)
and dropped in the finally (--keep-shadow retains it).

Seeding the target's platform/assumed schemas into a fresh co-located shadow (so
auth.users-style triggers resolve) is the next step; until then such dirs need an
explicit --shadow.

GREEN: 587 unit; co-located quick-mode integration pass; types + lint + knip clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6d393d1d4b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +131 to +135
if (char === "*" && next === "/") {
onSkipped?.("*/");
inBlockComment = false;
i += 2;
continue;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Track nested block comments in the SQL scanner

When --format-options processes valid PostgreSQL SQL with nested block comments, this state machine exits the comment at the first inner */ instead of tracking nesting. Tokens/commas in the rest of the outer comment are then treated as real SQL; for example CREATE TABLE t (a int /* outer /* inner */ , b int */ , c int) is formatted with b int */ outside the comment, producing invalid or semantically different DDL. Track block-comment depth here like the other literal/comment masking code does.

Useful? React with 👍 / 👎.

if (flags["keep-shadow"]) {
process.stderr.write(` Kept shadow database ${coLocated.name}\n`);
}
await coLocated.cleanup();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Let co-located shadows clean up on apply failure

When schema apply provisions a co-located shadow and the apply report is failed, the branch above calls process.exit(1), which exits without unwinding this finally; the same pattern applies to other post-provision process.exit gates. That leaves the newly created pgdelta_shadow_* database behind even though --keep-shadow was not requested, so use a return/throw or set process.exitCode so this cleanup runs before the command exits.

Useful? React with 👍 / 👎.

Comment on lines +765 to +767
...(flags["isolated-shadow"]
? { mode: "isolatedCluster" as const }
: {}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject isolated mode for co-located shadows

When --isolated-shadow is passed without --shadow, this line disables the shared-object leak check even though the shadow was just created co-located on the target cluster. The static cluster-DDL scan only recognizes direct role DDL, so a file can still run dynamic cluster changes (for example a DO block that CREATE ROLEs); with mode: "isolatedCluster" those target-cluster role/membership mutations are no longer detected. Reject --isolated-shadow unless an explicit dedicated --shadow is supplied, or keep databaseScratch mode for co-located shadows.

Useful? React with 👍 / 👎.

avallete and others added 7 commits July 2, 2026 19:51
…gence gaps

Five corpus scenarios + one rule unit test, each RED against the pre-fix
engine (captured failures):

- event-trigger-operations--replace-backing-function:
  "apply failed at action 0: cannot drop function ext.grant_access()
   because other objects depend on it" (both directions)
- type-operations--array-of-composite-column:
  forward: table/function created before the composite element type;
  reverse: "cannot drop type s.user_defined_filter because other objects
  depend on it"
- index-operations--standalone-unique-referenced-by-fk:
  "apply failed at action 3: there is no unique constraint matching given
   keys for referenced table \"tenants\"" — the plan contains no
  CREATE UNIQUE INDEX at all
- function-ops--replace-preserves-owner (isolatedCluster):
  drift: "unlink function:s.f() -> role:corpus_fn_owner /
  link function:s.f() -> role:test" — replace re-owns to the applier
- function-ops--replace-under-default-privileges (isolatedCluster):
  drift: "remove acl:(function:s.f()).corpus_adp_grantee" — recreate
  acquires an active-ADP grant the desired state never had
- extension-create.test.ts: non-relocatable extension in an independent
  schema must emit CREATE EXTENSION ... SCHEMA (bare CREATE before fix)

All surfaced by the sync-base-images zero-diff gate against the real
supabase/postgres:17.6.1.135 bare -> `supabase start` delta.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ng function is replaced

Functions are one opaque `def`, so any function change plans as a replace
(drop + recreate). The eventTrigger rule was missing `rebuildable`, so the
replacement closure skipped a surviving event trigger on a replaced
function and the plan emitted DROP FUNCTION while the trigger still
depended on it.

RED (corpus, prior commit):
  apply failed at action 0: cannot drop function ext.grant_access()
  because other objects depend on it
  0: DROP FUNCTION "ext"."grant_access"()

Surfaced by the Supabase baseline: extensions.grant_pg_net_access() and
five sibling access functions back standalone event triggers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lement type

A column/argument of type foo[] records its dependency against the implicit
array type _foo; the resolver's typ branch only matched typtype d/e/c/r, so
the edge to the composite/domain/enum element was silently dropped and the
table/function could sort before its type.

RED (corpus, prior commit): table with a user_defined_filter[] column
created before the CREATE TYPE (forward) / "cannot drop type
s.user_defined_filter because other objects depend on it" (reverse).

Surfaced by realtime.subscription.filters realtime.user_defined_filter[].

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…reign key

Index extraction excluded any index appearing in some constraint's
conindid, intending to skip PK/UNIQUE/EXCLUSION-owned indexes. But a
FOREIGN KEY also sets conindid — to the REFERENCED table's index — so a
standalone CREATE UNIQUE INDEX vanished from extraction the moment any FK
pointed at it. Gate the exclusion on contype IN ('p','u','x').

RED (corpus, prior commit):
  apply failed at action 3: there is no unique constraint matching given
  keys for referenced table "tenants"
  (no CREATE UNIQUE INDEX anywhere in the plan)

Surfaced by _realtime.tenants' unique index on external_id, referenced by
the FK from _realtime.extensions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…laces

Two emitter blind spots with the same root — a replace (drop + recreate)
behaves like a fresh create on the target, but two passes only covered
`added` facts:

1. Owner: the recreate re-owns the object as the applying role; owner is
   an edge re-emitted only from link/unlink deltas, and a replaced fact's
   owner is UNCHANGED (no delta) — so no ALTER ... OWNER TO was emitted and
   the object was silently re-owned to whoever ran the migration. Now every
   replaced fact (and replace-recreated descendant) re-establishes its
   owner from the projected target.
   RED drift: unlink function:s.f() -> role:corpus_fn_owner /
              link   function:s.f() -> role:test

2. Default-ACL hygiene: a recreate fires active ALTER DEFAULT PRIVILEGES
   exactly like a fresh create, acquiring grants the desired state never
   had (the source object predated the unchanged ADP). The hygiene pass
   (revoke ADP-injected grants with no desired acl fact) now also covers
   replaced facts.
   RED drift: remove acl:(function:s.f()).corpus_adp_grantee

Surfaced by the Supabase baseline: auth.uid()/role()/email() (owned by
supabase_auth_admin) reverted to the applier, and the replayed
extensions.grant_pg_net_access() carried a stale `postgres` grant from the
image's pre-existing default privileges.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…y-existing schemas

The SCHEMA clause was gated on extrelocatable alone, so a non-relocatable
extension installed into a pre-existing schema (pg_net into Supabase's
"extensions") got a bare CREATE EXTENSION and landed in the wrong schema,
leaving an unresolvable ALTER EXTENSION ... SET SCHEMA residual. Extraction
now records whether the extension OWNS its schema (pg_depend deptype 'e' on
the namespace, carried as non-semantic `_schemaIsMember`); CREATE emits
SCHEMA <s> whenever the schema exists independently and omits it only when
the extension creates its own.

RED: extension-create.test.ts (prior commit) — non-relocatable +
independent schema emitted a bare CREATE EXTENSION; end-to-end, the
baseline gate reported the pg_net SET SCHEMA residual.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ages)

Port the old package's sync-supabase-base-images to pg-delta-next,
dogfooding the engine end-to-end:

- scripts/sync-supabase-base-images.ts: stop local stacks, boot the bare
  pinned supabase/postgres image, `supabase start` a temp project pinned to
  the SAME tag (supabase/.temp/postgres-version + [db].major_version),
  diff bare -> full with extract()+plan(), render with renderPlanSql, write
  tests/fixtures/supabase-base-init/<major>.sql, then a ZERO-DIFF GATE:
  replay on a fresh bare container and require an empty re-plan. On batch
  failure, enumerateReplayFailures lists EVERY failing action;
  PGDELTA_SYNC_DEBUG_DIR dumps both gate-side snapshots for offline
  residual analysis. Wired as `bun run sync-base-images`.
- src/plan/render-sql.ts: renderPlanSql — preamble SETs + dependency-
  ordered action SQL as a flat replayable script (all-transactional plans).
- tests/supabase-base-init.ts: applySupabaseBaseInit(pool) replays the
  committed fixture (single batch, one connection).
- tests/supabase-base-init.test.ts: fast fixture-content guard pinning the
  convergence fixes without booting the Supabase stack.
- tests/fixtures/supabase-base-init/17.sql: generated baseline
  (965 actions, supabase/postgres:17.6.1.135) — zero-diff gate PASSED.

Dev tooling + test fixtures only (no published-behavior change): the six
engine fixes it surfaced carry their own changesets.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

if (sql[i] === "'" && sql[i + 1] === "'") i += 2;
else if (sql[i] === "'") {
i++;

P2 Badge Keep E-string escapes inside the masked literal

When a declarative file contains a PostgreSQL escape string (E'...') with an escaped quote, this scanner treats the \' quote as the end of the literal because it only recognizes doubled quotes. The remaining literal text is then scanned as real SQL, so a valid statement such as SELECT E'not\\'; COMMIT;'; can be rejected as transaction control, counted as extra statements for non-transactional files, or stripped by --skip-cluster-ddl; handle E-string backslash escapes here like the formatter scanner does.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +535 to +542
files = files.map((f) => {
const { kept, skipped } = stripClusterDdl(f.sql);
for (const s of skipped) {
process.stderr.write(
` SKIP cluster DDL (--skip-cluster-ddl) in ${f.name}: ${s.split("\n")[0]}\n`,
);
}
return { ...f, sql: kept };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Recheck for an empty desired state after skipping cluster DDL

When schema apply --scope database --skip-cluster-ddl is pointed at a directory whose only executable statements are role/membership DDL, the pre-skip hasExecutableSql guard passes, then this branch strips every statement and continues with an empty shadow. The fingerprint source is still the live target, so the resulting plan can apply destructive drops for all managed database objects; abort or re-run the non-empty/managed-fact check after stripClusterDdl removes statements.

Useful? React with 👍 / 👎.

Comment on lines 483 to 486
if (plain.length > 0) {
specs.push({
sql: `${defaultPrivPrefix(id)} ${verb} ${plain.join(", ")} ON ${objtype} TO ${grantee}`,
sql: `${defaultPrivPrefix(id)} GRANT ${plain.join(", ")} ON ${objtype} TO ${grantee}`,
consumes,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Revoke missing built-in default privileges before granting subsets

For a default-privilege fact that represents a built-in grantee with only a subset of its hard-wired default privileges, such as ALTER DEFAULT PRIVILEGES REVOKE UPDATE ON TABLES FROM owner, this create path only issues GRANT statements for the remaining privileges. Starting from a pristine database where the owner already has the full built-in table default, those grants are no-ops and the revoked privilege remains in effect for future objects; reset/revoke the built-in grant first when rendering a subset deviation.

Useful? React with 👍 / 👎.

Comment on lines +228 to +230
if (producerOf.has(childKey)) {
recreate(child.id);
continue;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mark inlined descendants as recreated by the parent replace

When a replaced parent create inlines a descendant via alsoProduces, this branch skips the duplicate standalone recreate but does not add the child to recreatedByReplace, so later set-deltas on that child still emit against the freshly recreated object. For example, replacing a partitioned table while changing an identity column's recorded sequence name produces CREATE TABLE ... AS IDENTITY and then ALTER SEQUENCE old_seq RENAME TO new_seq; old_seq was already dropped with the table, so the plan fails. Record these inlined children as recreated before recursing so their old set-deltas are suppressed.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant