Deterministic missing-marker catch + repair (follow-up to #224/#271, PR #273)

[pbean]

Follow-up to #224/#271 (PR #273, merged). The missing-marker fallback synthesizes a result when the bmad-dev-auto skill finalizes a spec's frontmatter to terminal status: without appending the ## Auto Run Result marker — but its attribution is heuristic (2-stable-Stops fingerprint, or single sighting post-kill), the spec is left non-compliant on disk, and PR #273 documented three accepted risks. This issue makes catch + fix deterministic with four harness-side mechanisms, in ONE PR, delivered in 4 single-session phases on branch feat/deterministic-marker-catch-<this issue>.

Mechanisms

1. M1 — launch-state snapshot + content-hash gate. _reset_spec_for_review (engine.py) captures SpecSnapshot(path, mtime_ns, sha256, fm_status) immediately after the [BUG] The #109 stall-grace fix cannot reach the review leg: a review pass is killed on its first Stop by the dev pass's stale ## Auto Run Result #160 marker strip and threads it through _run_session into a new SessionSpec.spec_snapshot field. _frontmatter_fallback deterministically refuses to synthesize when the candidate's bytes hash equals the snapshot (spec provably untouched by this session) — in ALL modes, including dead_window. Kills the documented dead-window false positive (review killed after an mtime-only bump, before flipping to in-review, scored done without running). New resultless verdict unmodified-since-launch; dead-window lifecycle crumb frontmatter-unmodified-refused. Snapshot is process-transient (crash-resume degrades to today's conservative path).
2. M2 — mid-session status-transition observation. New _observe_tick(handle, spec) no-op hook on CodingCLIAdapter (must be on the ABC — OpencodeDevAdapter shares _DevSynthesisMixin with its own wait loop), called from both heartbeat-throttled blocks (generic + opencode). The mixin impl reads the snapshot path's frontmatter each ~30s tick and records the FIRST observed normalized status that is non-empty, non-terminal, and ≠ the launch status (in practice in-review) into _fm_transition_obs, with lifecycle crumb spec-status-transition-observed. A recorded transition makes a later terminal frontmatter deterministic proof this session wrote it → sound single-sighting synthesis, live or dead window. Missed transitions (flip between ticks) fall back to the 2-obs fingerprint. Hash gate outranks transition evidence (bytes reverted to launch state must refuse).
3. M3 — artifact repair append_auto_run_result. New devcontract writer, inverse of strip_auto_run_result: appends ## Auto Run Result / Status: <fm status> / provenance note (ORCHESTRATOR_SYNTH_NOTE). False on absent file or existing real (non-fenced) heading; present-but-unreadable raises (caller imposes best-effort). Engine-side _repair_spec_marker(task, rj) called at the session-synthesized-from-frontmatter journal site (covers live-Stop, crash-path, and post-kill synthesis): guards = _generic_dev(), within-roots (skip reason out-of-tree), fm re-read must be terminal AND match rj["status"] (skip reason fm-mismatch — never author an inconsistent marker); journal kinds spec-marker-repaired / spec-marker-repair-failed / spec-marker-repair-skipped. Non-interference: _reconcile_generic_terminal_status only acts when fm LAGS prose; _salvage_review_timeout reads fm fresh; the append is engine-side ONLY (an adapter-side write would perturb its own mtime/hash observations).
4. M4 — targeted contract nudge, gated by new policy knob limits.dev_contract_nudge (bool, default true). At the first terminal-frontmatter-pending observation (wait=True, exactly one candidate, not dead-window, transition not proven), send one tmux nudge (CONTRACT_NUDGE_TEXT, parameterized spec path + status) telling the skill to append its required marker section, then end its turn. Exactly once per session via a never-cleared _contract_nudge_sent set, marked BEFORE the send (a raising transport still satisfies exactly-once); MultiplexerError swallowed; touches NO stall counters (Claude adapter: Stop hook + idle detection re-entrant loop prevents session from ending #149: an mtime bump resets the observation counter but must never re-nudge). A compliant skill's append is then harvested by the NORMAL marker scan; synthesis stays the backstop.

Decision ladder (_frontmatter_fallback, final shape)

scan → 0 candidates: pop obs, no-artifact → >1: ambiguous-frontmatter → read candidate (mtime, fm status, sha256 when snapshot path matches; OSError = torn-read degrade) → hash gate (M1, all modes) → evidence: transition_proven (M2) + fingerprint/stable/observations → synthesize when dead_window or transition_proven or (stable and obs >= 2) (crumb gains transition=) → record-obs branch: fingerprint + terminal-frontmatter-pending + contract nudge (M4).

HARD CONSTRAINT

The spec's status: done at review launch is load-bearing routing input to the upstream skill (step-01 → fresh step-04 review pass; _reset_spec_for_review docstring). Frontmatter status is NEVER mutated at review launch — all four mechanisms are observation or prose-append only.

Phases (one fresh session each, shared branch, PR opened in phase 4)

• S1 — M1: SpecSnapshot in adapters/base.py + engine capture/threading + hash gate in _frontmatter_fallback + adapter/engine tests + consolidated CHANGELOG entry
• S2 — M2: base _observe_tick hook + both heartbeat call sites (generic.py, opencode_http.py) + mixin observation + widened synthesis predicate + tests incl. opencode mirror
• S3 — M3: append_auto_run_result + ORCHESTRATOR_SYNTH_NOTE in devcontract.py + engine _repair_spec_marker + devcontract/engine tests
• S4 — M4: nudge + dev_contract_nudge knob (policy.py + core.toml + template + tests) + docs (FEATURES.md, tui-guide.md) + CHANGELOG finalize + open PR (ready-for-review, closes this issue) + file & cross-link the upstream BMAD-METHOD issue (bmad-dev-auto omits the marker append at review HALT)

Every phase ends: targeted pytest → full uv run pytest -q → trunk check --no-fix → commit → push (branch stays shippable).

Traps (carry into every phase)

• Do NOT add kwargs to _synth_result — the monkeypatched stub in tests/test_generic_tmux.py (def raising(handle, spec, *, wait, dead_window=False)) must stay valid; snapshot rides SessionSpec, state rides adapter dicts.
• _DevSynthesisMixin must declare send_text as a BARE class annotation, never a def (MRO: the mixin precedes the concrete adapter — a def would shadow the real send_text on GenericDevAdapter AND OpencodeDevAdapter).
• CHANGELOG Unreleased heading order is Added/Changed/Fixed; inserting a heading mid-section silently reparents entries. One consolidated entry, extended per phase.
• core.toml settings entries are default_ref/options_ref into policy.py, never literals; sync tests in tests/test_settings_schema.py enforce bidirectional coverage.
• Engine mock tests run the generic path (_generic_dev() always True): review runs via dev_effect(followup_review=True), sprint advances at DEV time.
• trunk check --no-fix (ruff F401 autofix deletes re-exports). uv run pytest (PEP668). 2 pre-existing test_module_skills_sync.py failures are known drift.
• First heartbeat tick always fires (last_heartbeat is None) — M2 observation runs on the first tick too.

[coderabbitai]

🔗 Related PRs

#53 - fix(adapters): complete dev sessions only on Stop or window death [merged]
#77 - feat(stories): spec-to-loop engine, adapter, CLI + HITL checkpoints (Phase 2 + 3) [merged]
#86 - feat(resolve): intent-gap patch-restore re-drive + inline-review-layer preflight (re-cut) [merged]
#95 - feat(adapters): post-kill reconcile rescues finished stalled/timeout sessions [merged]
#273 - fix(adapters,engine,escalation): synthesize marker-less terminal specs + review.on_timeout salvage [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[pbean]

✅ S1 — Mechanism 1 (launch-state snapshot + content-hash gate) landed

Branch feat/deterministic-marker-catch-276 @ 306882e (pushed; PR still deferred to S4).

What shipped

• SpecSnapshot(path, mtime_ns, sha256, fm_status) frozen dataclass in adapters/base.py, plus a new last SessionSpec.spec_snapshot field (positional constructions stay valid). Process-transient by design — no StoryTask.to_dict entry, so a crash-resume degrades to the conservative 2-observation path.
• engine._reset_spec_for_review now captures the snapshot immediately after the [BUG] The #109 stall-grace fix cannot reach the review leg: a review pass is killed on its first Stop by the dev pass's stale ## Auto Run Result #160 marker strip (bytes→sha256, st_mtime_ns, normalized fm status) and returns it; the capture is wrapped in try/except OSError → _journal_spec_read_failed(..., "review-launch-snapshot") → None while the strip keeps its raise-on-unreadable repair doctrine. The review-launch frontmatter status is never mutated (HARD CONSTRAINT reasserted in the docstring).
• _run_session(spec_snapshot=…) threads it into the SessionSpec; the review call site passes the captured snapshot.
• generic._frontmatter_fallback hash gate: when the single candidate is the snapshotted path and its bytes hash equals the snapshot, synthesis is deterministically refused in all modes — new resultless verdict unmodified-since-launch (wait=True), dead-window lifecycle crumb frontmatter-unmodified-refused, silent on the plain wait=False crash path. No observation recorded/popped. This kills the documented dead-window false positive (mtime-only bump before the in-review flip scored done without running).
• Consolidated Unreleased → Fixed CHANGELOG entry opened for the whole program ("Deterministic missing-marker catch + repair, Deterministic missing-marker catch + repair (follow-up to #224/#271, PR #273) #276"), with the Mechanism-1 line written (heading order Added/Changed/Fixed preserved).

Tests (all green)

• test_generic_tmux.py: …refuses_unmodified_snapshot_live, …_dead_window (post-kill returns the ORIGINAL result + lifecycle crumb), …hash_mismatch_still_two_obs, …snapshot_other_path_inert, …unmodified_wait_false_silent.
• test_engine.py: test_review_launch_snapshot_threaded_into_session_spec (dev snapshot None; review path/status/sha256 verified), test_review_launch_snapshot_degrades_on_unreadable_spec (directory-as-spec → None + spec-read-failed @ review-launch-snapshot).

Gates: uv run pytest tests/test_generic_tmux.py tests/test_engine.py -q → 331 passed; full uv run pytest -q → 2854 passed, 1 skipped, 2 pre-existing test_module_skills_sync.py drifts (bmad-loop-setup module.yaml, unrelated); trunk check --no-fix clean on all touched files.

Next: S2 — Mechanism 2 (mid-session status-transition observation).

[pbean]

S2 — M2 (mid-session status-transition observation) landed on feat/deterministic-marker-catch-276 (bd89dcf).

• New _observe_tick(self, handle, spec) -> None no-op hook on CodingCLIAdapter (base.py), called each heartbeat tick from both dev wait loops — GenericAdapter.wait_for_completion and OpencodeHttpAdapter.wait_for_completion — right after _write_heartbeat (the first tick always fires, last_heartbeat is None).
• _DevSynthesisMixin._observe_tick samples the snapshotted spec's frontmatter and records the first status this session drives off its launch state to a live, non-terminal value (in practice in-review) into _fm_transition_obs, with a spec-status-transition-observed lifecycle crumb. Pure sampling: needs a snapshot, fires at most once/session, silent on OSError; terminal (done/blocked), blank, and launch-status reads record nothing.
• _frontmatter_fallback computes transition_proven = snapshot present and str(path) == snap.path and task_id in _fm_transition_obs and widens synthesis to dead_window or transition_proven or (stable and observations >= FM_FALLBACK_MIN_OBS); the synthesized crumb gains transition=. The M1 hash gate stays above it — bytes reverted exactly to the launch snapshot still refuse (unmodified-since-launch) even with a transition recorded. A transition that flips entirely between two ticks is missed and falls back to the 2-observation fingerprint.
• Shared _DevSynthesisMixin means the hook is live on OpencodeDevAdapter too; a mirror e2e test asserts the OpenCode wait loop drives _observe_tick.
• CHANGELOG: M2 sub-bullet added to the consolidated Deterministic missing-marker catch + repair (follow-up to #224/#271, PR #273) #276 entry.

Tests (all green): 8 new in tests/test_generic_tmux.py (record-first-transition, ignore terminal/blank/launch-status via parametrize, no-snapshot/unreadable no-op, single-sighting harvest, missed-transition-stays-conservative, hash-gate-beats-transition, heartbeat-drives-observe-tick) + 1 opencode mirror. uv run pytest -q → 2865 passed, 1 skipped, 2 pre-existing test_module_skills_sync.py failures (confirmed present on the M1 base 306882e, unrelated bmad-loop-setup/assets/module.yaml drift). trunk check --no-fix clean.

Next: S3 — M3 (append_auto_run_result artifact repair).

• S1 — M1
• S2 — M2
• S3 — M3
• S4 — M4

[pbean]

S3 — M3 (artifact repair append_auto_run_result) landed on feat/deterministic-marker-catch-276 (29f77e6).

• New devcontract.append_auto_run_result(spec_path, status, *, detail="") -> bool — the inverse of strip_auto_run_result. Appends ## Auto Run Result / Status: <status> / the ORCHESTRATOR_SYNTH_NOTE provenance line (plus an optional detail paragraph). Returns False on an absent file or an existing real (non-fenced) marker heading (idempotence; a fence-quoted heading does NOT block — [BUG] find_result_artifact / parse_auto_run_result treat a fence-quoted heading as a real terminal section #52 symmetry); a present-but-unreadable spec RAISES (the repair-write doctrine strip/reset_spec_status share — the caller imposes best-effort). Before the append a marker-less terminal spec sits in find_frontmatter_candidates' territory (zero real headings); after it, in find_result_artifact's (≥1), so a later re-read is harvested on the normal marker path and the next review launch strips it exactly like a skill-written marker.
  • Newline handling: reads raw bytes (not read_text) so a CRLF spec's line endings are preserved (read_text's universal-newline translation would both hide the CRLF and silently rewrite the whole body to LF), ensures a trailing newline so the heading can never glue onto the last body line, and writes with newline="" (no platform re-translation on Windows). status is normalized lowercase and MUST equal the spec's frontmatter status — the caller passes exactly that — so synthesize_result's consistent cross-check holds on later re-reads.
• New engine _repair_spec_marker(task, rj), called immediately after the session-synthesized-from-frontmatter journal append in _run_session — the ONE site that covers live-Stop, crash-path, and post-kill dead-window synthesis (all carry the synthesized_from_frontmatter flag). Guards mirror _reconcile_generic_terminal_status: gated on _generic_dev(); the session-supplied spec_file must resolve to a real file within the orchestrator-owned roots (else spec-marker-repair-skipped reason out-of-tree); a fresh frontmatter re-read (_observed_frontmatter(..., "marker-repair")) must be terminal (done/blocked) and equal rj["status"] (else reason fm-mismatch — never author an inconsistent marker). Best-effort append: OSError → spec-marker-repair-failed; success → spec-marker-repaired (story_key, spec, status).
• Non-interference (in the docstring): _reconcile_generic_terminal_status only acts when the frontmatter LAGS prose — after this append the frontmatter is already terminal, so reconcile hits its idempotent/refusal branches; _salvage_review_timeout reads fm fresh and stays disjoint; the append is engine-side ONLY (an adapter-side write would perturb its own mtime/hash observation state — M1/M2).
• CHANGELOG: M3 sub-bullet added to the consolidated Deterministic missing-marker catch + repair (follow-up to #224/#271, PR #273) #276 entry.

Tests (all green): 8 new in tests/test_devcontract.py (minimal-shape + synthesize_result consistency, moves-to-marker-scan-territory, refuses-existing-real-heading, missing-file-false, raises-on-undecodable, over-fence-quoted-appends #52, missing-trailing-newline+CRLF preserved, append→strip byte-exact roundtrip) + 4 in tests/test_engine.py (full-run repair with provenance + spec-marker-repaired + summary.done==1; OSError best-effort → spec-marker-repair-failed; out-of-tree skip; fm-mismatch skip). uv run pytest -q → 2877 passed, 1 skipped, the 2 pre-existing test_module_skills_sync.py bmad-loop-setup/assets/module.yaml drifts (unrelated). trunk check --no-fix clean on all touched files.

Next: S4 — M4 (targeted contract nudge + limits.dev_contract_nudge knob + docs + open PR closing this issue + file the upstream BMAD-METHOD issue).

• S1 — M1
• S2 — M2
• S3 — M3
• S4 — M4

[pbean]

All four phases complete — PR #277 (ready for review)

Branch feat/deterministic-marker-catch-276, one PR, all four mechanisms landed:

• M1 — launch-state snapshot + content-hash gate (306882e): SpecSnapshot captured after the [BUG] The #109 stall-grace fix cannot reach the review leg: a review pass is killed on its first Stop by the dev pass's stale ## Auto Run Result #160 strip, threaded onto the review SessionSpec; the fallback refuses to synthesize from a byte-identical candidate in every mode. New verdict unmodified-since-launch / dead-window crumb frontmatter-unmodified-refused. Kills the dead-window false positive.
• M2 — mid-session status-transition observation (bd89dcf): _observe_tick on the ABC, called from both heartbeat loops (generic + opencode); records the first non-terminal transition off launch, crumb spec-status-transition-observed; widens synthesis to a single sighting when proven. Hash gate outranks it.
• M3 — artifact repair (29f77e6): devcontract.append_auto_run_result (inverse of the strip) + engine _repair_spec_marker at the session-synthesized-from-frontmatter site; journals spec-marker-repaired / -repair-failed / -repair-skipped; refuses on out-of-tree or fm-mismatch.
• M4 — targeted contract nudge (c5c85c2): new knob limits.dev_contract_nudge (bool, default true); one nudge per session on the first terminal-frontmatter-pending Stop, crumb contract-nudge-sent; exactly-once via a never-cleared set (marked before send), touches no stall counters. Synthesis stays the backstop.

HARD CONSTRAINT held throughout: the review-launch frontmatter status: is never mutated — every mechanism is observation or a prose-append.

Verification: full uv run pytest -q green (2887 passed; only the 2 pre-existing test_module_skills_sync.py bmad-loop-setup drift failures, unrelated), full trunk check --no-fix clean across all 15 modified files.

Upstream root cause filed: bmad-code-org/BMAD-METHOD#2616 — bmad-dev-auto intermittently omits the ## Auto Run Result append at the review HALT while finalizing terminal frontmatter, with the resultless-Stop evidence pattern and a request to make the HALT append unconditional-and-verified. This PR is the harness-side mitigation regardless of the upstream fix.

Closing via PR #277.