Add df.start_autonomous() for autonomous-transaction semantics

CHANGELOG.md

@@ -6,6 +6,10 @@ Pre-1.0 note: while `pg_durable` is in major version `0`, minor releases may inc
 
 ## [0.2.4] - Unreleased
 
+### Added
+
+- **`df.start_autonomous()` — Oracle-style autonomous transactions:** a new function that starts a durable function with autonomous-transaction semantics. It persists and enqueues the work on a separate PostgreSQL session, so it commits independently and **survives a rollback of the caller's transaction** (the PostgreSQL equivalent of Oracle's `PRAGMA AUTONOMOUS_TRANSACTION`). Use it for audit trails, error logging, and other "must persist even if the caller rolls back" work. Ordinary `df.start()` is unchanged and still participates in the caller's transaction. Takes the same arguments as `df.start()` (`fut`, optional `label`, optional `database`) and runs under the calling role's identity. Added additively as a new C symbol, so the new `.so` stays backward compatible with all previous schemas in this provider line (see `docs/upgrade-testing.md`).
+
 ### Changed
 
 - **`df.wait_for_schedule()` cron timing:** the next cron tick is now computed at execution time using duroxide's deterministic clock (`ctx.utc_now()`) inside the `execute_function_graph` orchestration, instead of being pre-computed at `df.start()` time. This makes recurring `@>` schedules and any start-to-execution delay target the correct upcoming tick (#130).

USER_GUIDE.md

@@ -152,8 +152,37 @@ SELECT 'SELECT 1' ~> 'SELECT 2';
 SELECT df.start('SELECT 1' ~> 'SELECT 2');
 ```
 
+### Transaction Semantics & Autonomous Transactions
+
+`df.start()` participates in the **caller's transaction**: the instance and its
+graph are written via the caller's session, so if the caller's transaction
+rolls back, the durable function is rolled back with it and never runs.
+
+`df.start_autonomous()` provides the opposite guarantee — the PostgreSQL
+equivalent of Oracle's `PRAGMA AUTONOMOUS_TRANSACTION`. It persists and enqueues
+the durable function on a **separate session**, so it commits independently and
+**survives a rollback of the caller's transaction**. Use it for audit trails,
+error logging, and other "must persist even if the caller rolls back" work.
+
+```sql
+BEGIN;
+    INSERT INTO employees (id, name) VALUES (999, 'Test User');
+
+    -- Persists even though the surrounding transaction rolls back below.
+    SELECT df.start_autonomous(
+        'INSERT INTO audit_log (message) VALUES (''Inserted employee 999'')',
+        'audit'
+    );
+
+    ROLLBACK;  -- employees insert is undone; the audit_log entry remains
+```
+
+`df.start_autonomous()` takes the same arguments as `df.start()` and runs under
+the calling role's identity and privileges.
+
 ---
 
+
 ## DSL Reference
 
 ### Auto-Wrap SQL
@@ -182,6 +211,7 @@ df.sql('SELECT 1') ~> df.sql('SELECT 2')
 | `df.break()` | Exit enclosing loop | `df.break()` |
 | `df.break(value)` | Exit loop with **literal** return value (not auto-wrapped as SQL) | `df.break('{"done": true}')` |
 | `df.start(func, label, database)` | Start function (optionally in another database) | `df.start('SELECT 1', 'job')` |
+| `df.start_autonomous(func, label, database)` | Start function with autonomous-transaction semantics (survives caller rollback) | `df.start_autonomous('INSERT INTO audit ...', 'audit')` |
 | `df.cancel(id, reason)` | Cancel function | `df.cancel('a1b2c3d4', 'Done')` |
 | `df.status(id)` | Get status by instance_id (not label) | `df.status('a1b2c3d4')` |
 | `df.result(id)` | Get result by instance_id (not label) | `df.result('a1b2c3d4')` |

docs/upgrade-testing.md

@@ -205,6 +205,13 @@ what the upgrade script handles, and any backward compatibility considerations.
 
 ### v0.2.3 → v0.2.4
 
+#### Add `df.start_autonomous()` — Oracle-style autonomous transactions
+- **DDL change (df schema):** Adds a new SQL-callable function `df.start_autonomous(text, text, text)` (bound to the new C symbol `start_autonomous_wrapper`). It persists and enqueues a durable function on a *separate* PostgreSQL session so it commits independently and survives a rollback of the caller's transaction — the PostgreSQL equivalent of Oracle's `PRAGMA AUTONOMOUS_TRANSACTION`. Ordinary `df.start(text, text, text)` is unchanged and still participates in the caller's transaction.
+- **Upgrade script:** `sql/pg_durable--0.2.3--0.2.4.sql` runs a single `CREATE FUNCTION df.start_autonomous(...)`, copied verbatim from the pgrx-generated fresh-install DDL (same argument list, defaults, `RETURNS TEXT`, `LANGUAGE c`, and wrapper symbol). New `df.*` functions retain PostgreSQL's default PUBLIC `EXECUTE`, gated by `USAGE ON SCHEMA df`, so no explicit `GRANT` is needed.
+- **Scenario A considerations:** The upgrade path creates exactly the same `df.start_autonomous` overload as a fresh install (identical signature and wrapper binding), so the Scenario A snapshot matches.
+- **Scenario B1 considerations:** Pure additive change — a brand-new function on a new symbol; every pre-existing function and its C symbol is untouched. The new `.so` therefore runs correctly against all previous schemas in this provider line (0.2.2, 0.2.3); an un-upgraded schema simply does not expose `df.start_autonomous` yet, and `df.start()` keeps binding to the unchanged `start_wrapper`. `df.start_autonomous()` reads/writes only columns (`df.instances`, `df.nodes`) that exist in every shipped schema on this line.
+- **Scenario B2 considerations:** No data migration; instances created before the upgrade are unaffected.
+
 #### Simplify `df.grant_usage()` — drop the explicit function allowlist
 - **DDL change (df schema):** `df.grant_usage()` no longer loops over a hard-coded `func_sigs` array issuing `GRANT EXECUTE` per function. Fresh installs (`src/lib.rs`) and the upgrade script (`sql/pg_durable--0.2.3--0.2.4.sql`) both `CREATE OR REPLACE` the function with a body that grants `USAGE ON SCHEMA df` plus the table privileges, and conditionally grants `df.http()` / the admin helpers. The signature `df.grant_usage(text, boolean, boolean)` is unchanged.
 - **DDL change (df schema):** `df.revoke_usage()` is made symmetric with the new `grant_usage()`. It no longer loops over every `df.*` function in `pg_proc` issuing `REVOKE EXECUTE` (which, post-simplification, only produced "no privileges could be revoked" warnings since ordinary functions are never granted per-function EXECUTE). The new body revokes only what `grant_usage()` grants: schema `USAGE`, EXECUTE on the sensitive functions (`df.http`, `df.grant_usage`, `df.revoke_usage`), and the table privileges. The signature `df.revoke_usage(text)` is unchanged.

sql/pg_durable--0.2.3--0.2.4.sql

@@ -391,3 +391,32 @@ CREATE  FUNCTION df."list_instances"(
 
 LANGUAGE c /* Rust */
 AS 'MODULE_PATHNAME', 'list_instances_paged_wrapper';
+
+-- ============================================================================
+-- Add df.start_autonomous(): Oracle-style autonomous transaction support.
+--
+-- df.start_autonomous() persists and enqueues the durable function on a
+-- separate PostgreSQL session, so it commits independently and survives a
+-- rollback of the caller's transaction (the PostgreSQL equivalent of Oracle's
+-- PRAGMA AUTONOMOUS_TRANSACTION). Ordinary df.start() is unchanged and still
+-- participates in the caller's transaction.
+--
+-- Pure additive change: a brand-new function bound to a new C symbol
+-- (start_autonomous_wrapper). Existing functions and their symbols are
+-- untouched, so the new .so stays backward compatible with all previous
+-- schemas in this provider line (Scenario B1) — an un-upgraded schema simply
+-- does not expose df.start_autonomous yet. The CREATE FUNCTION block is the
+-- pgrx-generated fresh-install DDL for src/dsl.rs::start_autonomous copied
+-- verbatim, so the Scenario A snapshot matches a fresh 0.2.4 install. New df.*
+-- functions retain PostgreSQL's default PUBLIC EXECUTE (gated by USAGE ON
+-- SCHEMA df), so no explicit GRANT is needed.
+-- ============================================================================
+-- pg_durable::dsl::start_autonomous
+CREATE  FUNCTION df."start_autonomous"(
+	"fut" TEXT, /* &str */
+	"label" TEXT DEFAULT NULL, /* core::option::Option<&str> */
+	"database" TEXT DEFAULT NULL /* core::option::Option<&str> */
+) RETURNS TEXT /* alloc::string::String */
+
+LANGUAGE c /* Rust */
+AS 'MODULE_PATHNAME', 'start_autonomous_wrapper';

src/client.rs

@@ -16,7 +16,9 @@ use duroxide::Client;
 use pgrx::prelude::*;
 use tokio::runtime::Runtime;
 
-use crate::types::{backend_duroxide_schema, new_backend_provider, postgres_connection_string};
+use crate::types::{
+    backend_duroxide_schema, connect_as_user, new_backend_provider, postgres_connection_string,
+};
 
 /// Cached tokio runtime for client operations.
 static CLIENT_RUNTIME: OnceLock<Runtime> = OnceLock::new();
@@ -220,6 +222,72 @@ pub fn start_durable_function(
     })
 }
 
+/// Start a durable function with **autonomous transaction** semantics.
+///
+/// This provides the Oracle `PRAGMA AUTONOMOUS_TRANSACTION` equivalent: the
+/// durable function's graph is persisted and enqueued on a *separate*
+/// PostgreSQL session, so it commits independently and **survives a rollback of
+/// the caller's transaction**.
+///
+/// Mechanism: open a fresh loopback connection authenticated as `user` and run
+/// the ordinary `df.start(...)` there. Because sqlx runs each statement in
+/// autocommit mode, that inner `df.start` commits in its own transaction the
+/// moment it returns — regardless of what the outer caller later does. This is
+/// the same "separate backend" technique `pg_background` uses to achieve
+/// autonomy, but it reuses pg_durable's existing graph construction and
+/// duroxide enqueue path unchanged.
+///
+/// Returns the new instance id.
+///
+/// FUTURE EXPLORATION (alternative implementation): instead of opening another
+/// client connection just to run `df.start` in a new backend, move the graph
+/// persistence into the parent orchestration itself. With autonomy, the full
+/// function graph would be encoded directly into the `start_orchestration`
+/// input payload (which duroxide already commits out-of-band), and the
+/// orchestration's first activity would become a `store-function-graph`
+/// activity that writes `df.instances` / `df.nodes` from the background worker
+/// — replacing today's `load-function-graph` first activity. That would remove
+/// the extra backend connection entirely and collapse the current dual write
+/// (graph in the caller's txn, enqueue out-of-band) into a single durable
+/// source of truth, eliminating the orphaned-orchestration race for good. See
+/// the PR description for the trade-offs worth evaluating.
+pub fn start_autonomous(
+    fut: &str,
+    label: Option<&str>,
+    database: Option<&str>,
+    user: &str,
+) -> Result<String, String> {
+    use sqlx::Row;
+
+    let rt = get_client_runtime();
+
+    let fut = fut.to_string();
+    let label = label.map(|s| s.to_string());
+    let database = database.map(|s| s.to_string());
+    let user = user.to_string();
+
+    rt.block_on(async {
+        // Connect to the *current* database (database=None resolves to the
+        // control/target database) as the submitting role, so identity and
+        // privilege checks inside the inner df.start run as the caller.
+        let mut conn = connect_as_user(&user, None).await?;
+
+        let row = sqlx::query("SELECT df.start($1, $2, $3) AS id")
+            .bind(&fut)
+            .bind(&label)
+            .bind(&database)
+            .fetch_one(&mut conn)
+            .await
+            .map_err(|e| format!("autonomous df.start failed: {e}"))?;
+
+        let id: String = row
+            .try_get("id")
+            .map_err(|e| format!("autonomous df.start returned no instance id: {e}"))?;
+
+        Ok(id)
+    })
+}
+
 /// Cancel a durable function.
 pub fn cancel_durable_function(instance_id: &str, reason: &str) -> Result<(), String> {
     let inst_id = instance_id.to_string();

src/dsl.rs

@@ -1074,6 +1074,42 @@ pub fn start(
     instance_id
 }
 
+/// Starts a durable SQL function with **autonomous transaction** semantics.
+///
+/// This is the PostgreSQL equivalent of Oracle's `PRAGMA AUTONOMOUS_TRANSACTION`:
+/// the durable function is persisted and enqueued on a *separate* PostgreSQL
+/// session, so it commits independently and **survives a rollback of the
+/// caller's transaction**. Use it for audit/error logging and other
+/// "must-persist-even-if-the-caller-rolls-back" work.
+///
+/// Ordinary [`start`] participates in the caller's transaction: if the caller
+/// rolls back, the durable function is rolled back with it. `start_autonomous`
+/// is the opposite — it always persists once this call returns.
+///
+/// Accepts the same arguments as [`start`] (`fut`, optional `label`, optional
+/// `database`) and returns the new instance id.
+#[pg_extern(schema = "df")]
+pub fn start_autonomous(
+    fut: &str,
+    label: default!(Option<&str>, "NULL"),
+    database: default!(Option<&str>, "NULL"),
+) -> String {
+    // Capture the calling role so the separate session runs df.start() with the
+    // same identity (and therefore the same privileges / RLS scope).
+    let user = unsafe {
+        let oid = pgrx::pg_sys::GetUserId();
+        let name_ptr = pgrx::pg_sys::GetUserNameFromId(oid, false);
+        std::ffi::CStr::from_ptr(name_ptr)
+            .to_string_lossy()
+            .into_owned()
+    };
+
+    match crate::client::start_autonomous(fut, label, database, &user) {
+        Ok(id) => id,
+        Err(e) => pgrx::error!("{}", e),
+    }
+}
+
 /// Cancels a running durable function.
 #[pg_extern(schema = "df")]
 pub fn cancel(instance_id: &str, reason: default!(&str, "'Cancelled by user'")) -> String {

tests/e2e/sql/54_autonomous_transaction.sql

@@ -0,0 +1,116 @@
+-- Copyright (c) Microsoft Corporation.
+-- Licensed under the PostgreSQL License.
+
+-- Oracle-style autonomous transaction support via df.start_autonomous().
+--
+-- Demonstrates that df.start_autonomous(...) commits the durable function
+-- independently of the caller's transaction: it SURVIVES a caller ROLLBACK
+-- (like Oracle PRAGMA AUTONOMOUS_TRANSACTION), whereas the default df.start()
+-- is rolled back with the caller.
+SET SESSION AUTHORIZATION df_e2e_user;
+
+DROP TABLE IF EXISTS test_autonomous_audit;
+CREATE TABLE test_autonomous_audit (id SERIAL, message TEXT);
+
+DROP TABLE IF EXISTS test_autonomous_main;
+CREATE TABLE test_autonomous_main (id INT);
+
+-- === Part 1: autonomous => true SURVIVES a caller rollback ===
+
+BEGIN;
+    -- Main-transaction work that will be rolled back.
+    INSERT INTO test_autonomous_main (id) VALUES (999);
+
+    -- Autonomous durable function: commits on a separate session.
+    SELECT df.start_autonomous(
+        'INSERT INTO test_autonomous_audit (message) VALUES (''logged from autonomous txn'')',
+        'test-autonomous-survives'
+    );
+
+    -- Simulate a failure in the surrounding transaction.
+    ROLLBACK;
+
+DO $$
+DECLARE
+    inst_id     TEXT;
+    status      TEXT;
+    main_count  INT;
+    audit_count INT;
+BEGIN
+    -- The main-transaction insert must have been rolled back.
+    SELECT count(*) INTO main_count FROM test_autonomous_main;
+    IF main_count <> 0 THEN
+        RAISE EXCEPTION 'TEST FAILED: main insert should have rolled back, got % rows', main_count;
+    END IF;
+
+    -- The autonomous instance must have survived the rollback.
+    SELECT id INTO inst_id
+    FROM df.instances
+    WHERE label = 'test-autonomous-survives'
+    ORDER BY created_at DESC
+    LIMIT 1;
+
+    IF inst_id IS NULL THEN
+        RAISE EXCEPTION 'TEST FAILED: autonomous instance did not survive caller rollback';
+    END IF;
+
+    SELECT df.await_instance(inst_id) INTO status;
+    IF status <> 'completed' THEN
+        RAISE EXCEPTION 'TEST FAILED: autonomous instance status = %', status;
+    END IF;
+
+    -- The audit row must have persisted independently of the rollback.
+    SELECT count(*) INTO audit_count
+    FROM test_autonomous_audit
+    WHERE message = 'logged from autonomous txn';
+
+    IF audit_count <> 1 THEN
+        RAISE EXCEPTION 'TEST FAILED: audit row missing, count = % (autonomous txn did not persist)', audit_count;
+    END IF;
+
+    RAISE NOTICE 'PASSED: autonomous => true survived caller rollback';
+END $$;
+
+-- === Part 2: default df.start() does NOT survive rollback ===
+
+BEGIN;
+    SELECT df.start(
+        'INSERT INTO test_autonomous_audit (message) VALUES (''should never persist'')',
+        'test-autonomous-transactional'
+    );
+    ROLLBACK;
+
+DO $$
+DECLARE
+    inst_count  INT;
+    audit_count INT;
+BEGIN
+    -- The instance row was written via SPI in the caller's transaction, so the
+    -- rollback removes it. (A dangling duroxide orchestration may briefly exist
+    -- and then fail to load the graph — it never runs the SQL.)
+    SELECT count(*) INTO inst_count
+    FROM df.instances
+    WHERE label = 'test-autonomous-transactional';
+
+    IF inst_count <> 0 THEN
+        RAISE EXCEPTION 'TEST FAILED: transactional instance should not survive rollback, found %', inst_count;
+    END IF;
+
+    -- Give any dangling orchestration a moment; the SQL must never run.
+    PERFORM pg_sleep(1);
+
+    SELECT count(*) INTO audit_count
+    FROM test_autonomous_audit
+    WHERE message = 'should never persist';
+
+    IF audit_count <> 0 THEN
+        RAISE EXCEPTION 'TEST FAILED: transactional df.start persisted work across rollback, count = %', audit_count;
+    END IF;
+
+    RAISE NOTICE 'PASSED: default df.start() rolled back with the caller';
+END $$;
+
+DROP TABLE test_autonomous_audit;
+DROP TABLE test_autonomous_main;
+
+SELECT 'TEST PASSED' AS result;