Skip to content

Add df.start_autonomous() for autonomous-transaction semantics#285

Open
pinodeca wants to merge 1 commit into
mainfrom
feat/autonomous-transactions
Open

Add df.start_autonomous() for autonomous-transaction semantics#285
pinodeca wants to merge 1 commit into
mainfrom
feat/autonomous-transactions

Conversation

@pinodeca

@pinodeca pinodeca commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds df.start_autonomous() — the PostgreSQL equivalent of Oracle's PRAGMA AUTONOMOUS_TRANSACTION. It starts a durable function that commits independently and survives a rollback of the caller's transaction. Ordinary df.start() is unchanged and still participates in the caller's transaction.

This closes the specific gap raised when comparing pg_durable to pg_background: the "must persist even if the parent rolls back" requirement (audit trails, error logging, etc.). Everything else pg_durable already runs autonomously once the caller commits — this adds the rollback-survival case.

BEGIN;
    INSERT INTO employees (id, name) VALUES (999, 'Test User');

    -- Persists even though the surrounding transaction rolls back below.
    SELECT df.start_autonomous(
        'INSERT INTO audit_log (message) VALUES (''Inserted employee 999'')',
        'audit'
    );

    ROLLBACK;  -- employees insert is undone; the audit_log entry remains

How it works

df.start_autonomous() opens a fresh loopback connection authenticated as the calling role and runs the ordinary df.start(...) there. Because each statement runs in autocommit mode, that inner df.start commits in its own transaction the moment it returns — regardless of what the outer caller later does. This is the same "separate backend" technique pg_background uses, but it reuses pg_durable's existing graph construction and duroxide enqueue path unchanged, and it avoids the orphaned-orchestration race that the default path has when a caller rolls back after df.start().

df.start_autonomous() runs under the caller's identity and privileges (same RLS scope), and takes the same arguments as df.start() (fut, optional label, optional database).

Backward compatibility (upgrade)

Folded into the unreleased 0.2.4 (latest shipped tag is v0.2.3), so no new version was cut.

  • Additive only: a brand-new function bound to a new C symbol (start_autonomous_wrapper). No existing function or symbol changes.
  • Scenario A: the 0.2.3 → 0.2.4 upgrade script's CREATE FUNCTION is the pgrx-generated fresh-install DDL copied verbatim, so upgrade and fresh install produce identical catalogs.
  • Scenario B1: the new .so runs correctly against all previous schemas in the provider line (0.2.2, 0.2.3) — df.start() keeps binding to the unchanged start_wrapper; an un-upgraded schema simply doesn't expose df.start_autonomous yet.
  • Scenario B2: no data migration.

⚠️ I intentionally did not change df.start()'s arity. Adding a parameter to the existing function would make the new .so read an out-of-bounds argument when running against an un-upgraded (old-SQL) schema — a B1 violation given pg_durable's decoupled .so/schema deployment model. A separate function is the B1-safe design.

Future exploration (please weigh in)

There's a note in src/client.rs proposing a cleaner alternative to opening a second client connection just to run df.start in a new backend:

Instead, move graph persistence into the parent orchestration itself. With autonomy, the full function graph would be encoded directly into the start_orchestration input payload (which duroxide already commits out-of-band), and the orchestration's first activity would become a store-function-graph activity that writes df.instances / df.nodes from the background worker — replacing today's load-function-graph first activity.

That would remove the extra backend connection entirely and collapse the current dual write (graph in the caller's txn, enqueue out-of-band) into a single durable source of truth, eliminating the orphaned-orchestration race for good. Worth evaluating the trade-offs (payload size limits for large graphs, where identity/privilege validation happens, and how ordinary non-autonomous df.start() would share the path).

Testing

  • New E2E test tests/e2e/sql/54_autonomous_transaction.sql proves (a) df.start_autonomous() survives a caller ROLLBACK and persists its work, and (b) default df.start() rolls back with the caller.
  • ./scripts/test-unit.sh — 194 passed
  • ./scripts/test-e2e-local.sh 54_autonomous — passed
  • ./scripts/test-upgrade.sh — all passed (Scenario A / B1 / B2)
  • cargo fmt --check + cargo clippy — clean

Adds df.start_autonomous(), the PostgreSQL equivalent of Oracle's
PRAGMA AUTONOMOUS_TRANSACTION. It persists and enqueues a durable
function on a separate PostgreSQL session, so it commits independently
and survives a rollback of the caller's transaction. Ordinary df.start()
is unchanged and still participates in the caller's transaction.

- src/dsl.rs: new df.start_autonomous(fut, label, database) pg_extern
- src/client.rs: start_autonomous() loopback helper (+ note on a future
  alternative implementation using a store-function-graph activity)
- sql/pg_durable--0.2.3--0.2.4.sql: additive CREATE FUNCTION (new C
  symbol; backward compatible with all prior schemas in the provider line)
- docs/upgrade-testing.md, USER_GUIDE.md, CHANGELOG.md: documentation
- tests/e2e/sql/54_autonomous_transaction.sql: proves survival across
  caller rollback, and that default df.start() rolls back with the caller
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant