Skip to content

feat(durable): journal writer actor, sealed backend, local durable.db backend (#4946)#4968

Merged
bug-ops merged 1 commit into
mainfrom
feat/m28/4946-durable-writer-backend-local
Jun 6, 2026
Merged

feat(durable): journal writer actor, sealed backend, local durable.db backend (#4946)#4968
bug-ops merged 1 commit into
mainfrom
feat/m28/4946-durable-writer-backend-local

Conversation

@bug-ops

@bug-ops bug-ops commented Jun 6, 2026

Copy link
Copy Markdown
Owner

Summary

Implements C3 of the durable execution epic (#4707, spec-064): the persistence engine behind a durable execution. Builds on C2 (#4967, now merged). Part of milestone M28.

Closes #4946.

What lands

  • LocalBackend (backend/local.rs) — owns a dedicated durable.db pool (INV-14; schema applied via zeph_db::run_migrations, no .sql/sqlx::migrate! in this crate). Implements the Journal trait: append (single, RETURNING seq), read_execution, read_execution_range, finalize (via begin_write), and a prune stub. AEAD-seals payload-bearing entries through an injected Option<Arc<dyn PayloadCipher>> with the entry location bound as associated data; stamps a keyed-BLAKE3 row HMAC over control entries when an HMAC key is configured. Enforces max_payload_bytes on both write and read (INV-11, fail-closed).
  • JournalWriter actor + JournalWriterHandle (writer.rs) — decouples DB writes from the calling path:
    • buffered appends group-commit on journal_flush_interval_ms and drop with a WARN on a full channel (re-run safely on resume);
    • exactly-once appends flush all causally-preceding buffered entries before committing (INV-4) and return their JournalSeq over a oneshot bounded by journal_ack_timeout_ms (INV-12 / FR-DE-11 — never blocks indefinitely);
    • resumes from MAX(seq) on (re)start (FR-DE-12);
    • emits the durable.journal.writer.queue_depth gauge per commit cycle.
  • Sealed ExecutionBackend + BackendCapabilities + DurableBackendEnum (backend.rs) — enum dispatch, no Box<dyn> on the hot path; the trait is sealed (un-implementable outside the crate), so open/resolve_promise/due_timers join it in C4/C5 without breaking callers.

Scope boundaries

  • Promise, timer, and checkpoint journal entries fail closed with DurableError::UnsupportedEntryKind — their persistence (and any schema additions) lands with the promise/timer (C5) and retention layers, which the durable_journal columns do not yet cover.
  • DurableContext/DurableStep, the replay cursor, and the daemon TaskSupervisor wiring are C4.

Spans

durable.backend.open, durable.journal.append, durable.journal.read, durable.journal.read_segment, durable.journal.finalize, durable.journal.prune, and the durable.journal.writer.queue_depth gauge.

Tests

50 tests pass. Backend + writer integration tests are SQLite-gated (:memory:, mirroring zeph-scheduler); the handle ACK-timeout / buffered-drop tests are dialect-agnostic.

  • cargo nextest run -p zeph-durable --no-default-features --features sqlite — 50 passed
  • dual-backend compile: cargo check -p zeph-durable under sqlite, postgres, and both
  • cargo clippy --all-targets -- -D warnings clean under sqlite and postgres
  • cargo test --doc (23) + RUSTDOCFLAGS="--deny rustdoc::broken_intra_doc_links" cargo doc --no-deps clean
  • RUSTFLAGS="-D warnings" cargo check --all-targets clean; downstream zeph-core still compiles

Acceptance criteria

  • INV-14: dedicated durable.db pool + migrations via zeph_db::run_migrations
  • INV-4 / FR-DE-04: flush-before-commit for exactly-once intents
  • INV-12 / FR-DE-11: AppendAcked times out to JournalUnavailable; caller never blocks
  • INV-12 / FR-DE-12: MAX(seq) restart resume, no gap/duplication
  • full-channel buffered drop (WARN) vs acked wait
  • system-invariants §13: dual-dialect SQL (sql!(), i64/Vec<u8>) compiles under both backends
  • ExecutionBackend sealed; DurableBackendEnum enum dispatch (no Box<dyn>)
  • spans present (durable.journal.append, durable.backend.open, durable.journal.writer.queue_depth)

@github-actions github-actions Bot added documentation Improvements or additions to documentation rust Rust code changes dependencies Dependency updates enhancement New feature or request size/XL Extra large PR (500+ lines) labels Jun 6, 2026
@bug-ops bug-ops self-assigned this Jun 6, 2026
… backend (#4946)

Add the durable persistence engine to zeph-durable (spec-064 C3):

- LocalBackend owns a dedicated durable.db pool (INV-14, schema via
  zeph_db::run_migrations), implements the Journal trait (append with
  RETURNING seq, full and ranged reads, finalize via begin_write, prune
  stub), AEAD-seals payload-bearing entries through an injected
  Option<Arc<dyn PayloadCipher>> with the entry location bound as AAD, and
  stamps a keyed-BLAKE3 row HMAC over control entries when a key is set.
  Enforces max_payload on both write and read (INV-11).
- JournalWriter background actor decouples writes from the calling path:
  buffered appends group-commit on a flush interval (dropped with WARN
  under backpressure), exactly-once appends flush all causally-preceding
  entries before committing (INV-4) and return their JournalSeq over a
  oneshot bounded by journal_ack_timeout_ms (INV-12, FR-DE-11), and the
  writer resumes from MAX(seq) on restart (FR-DE-12). Emits the
  durable.journal.writer.queue_depth gauge per commit cycle.
- Sealed ExecutionBackend trait + BackendCapabilities + DurableBackendEnum
  enum dispatch keep the backend surface closed with no Box<dyn> on the hot
  path. open/resolve_promise/due_timers join the sealed trait with C4/C5.

Promise, timer, and checkpoint journal entries fail closed with
DurableError::UnsupportedEntryKind until the promise/timer/retention layers
land. All SQL uses sql!() and dialect-agnostic i64/Vec<u8> column types;
compiles under sqlite, postgres, and both. 50 unit/integration tests pass.
@bug-ops bug-ops force-pushed the feat/m28/4946-durable-writer-backend-local branch from 2b0a186 to cb868e3 Compare June 6, 2026 22:07
@bug-ops bug-ops enabled auto-merge (squash) June 6, 2026 22:08
@bug-ops bug-ops merged commit a7473b7 into main Jun 6, 2026
36 checks passed
@bug-ops bug-ops deleted the feat/m28/4946-durable-writer-backend-local branch June 6, 2026 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Dependency updates documentation Improvements or additions to documentation enhancement New feature or request rust Rust code changes size/XL Extra large PR (500+ lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(durable): journal writer actor, sealed backend, local durable.db backend

1 participant