fix(bin): enforce clone identity at spawn and verify recorded worktree on recovery resume - #878
fix(bin): enforce clone identity at spawn and verify recorded worktree on recovery resume#878sparkus wants to merge 3 commits into
Conversation
|
Additional field evidence from the originating incident (post-submission): after a foreign-slot rejection, a stale TREEHOUSE_DIR environment variable in the recovered session still pointed at the rejected foreign slot; only a marker file was touched before detection, but it shows worktree rejection/reallocation must also REBIND the session environment (TREEHOUSE_DIR and any worktree-derived env), not just the cwd. Worth covering in the post-resume verification this PR adds. |
e5aa1c8 to
a63f0df
Compare
|
Automated reminder: thanks for the PR! This branch currently has a merge conflict with the base branch. When you get a chance, please rebase onto (or merge) the latest base branch, resolve the conflict, and push. After that, checks will re-run and the PR will get looked at again. Noted for firstmate#878 at |
a63f0df to
2d26711
Compare
|
Automated reminder: thanks for the PR! This branch currently has a merge conflict with the base branch. When you get a chance, please rebase onto (or merge) the latest base branch, resolve the conflict, and push. After that, checks will re-run and the PR will get looked at again. Noted for firstmate#878 at |
Intent
Fix the confirmed cross-home worktree-pool isolation defect in Firstmate's shared tooling. A ship/scout spawn must canonicalize Git common-directory identity for the selected project clone and proposed worktree, reject a foreign home's pooled worktree before hooks, agent launch, or task metadata publication, and record the selected clone identity for recovery. Recovery or harness resume paths that can restore historical cwd, especially Grok resume, must verify the live agent cwd, Git top-level, and repository identity against the recorded worktree before any project mutation, failing closed and requiring a fresh launch when they differ. Add regressions using two independent clones of the same remote under one shared pool namespace and a simulated historical-cwd restoration. Keep the fix Firstmate-side without changing Treehouse, preserve supported backend behavior, keep shell scripts shellcheck-clean, use one sentence per line in Markdown, avoid incident restatement beyond the operational contract, base and target main, and never add an agent co-author.
What Changed
bin/fm-spawn.shnow canonicalizes the Git common-directory identity of the selected project clone and the proposed worktree (via newbin/fm-worktree-identity-lib.sh), rejects a foreign home's pooled worktree before any hooks, agent launch, or task metadata publication, and records the selected clone identity asproject_git_common_dir=in task metadata for recovery.bin/fm-recovery-verify.shchecks a resumed task's live agent cwd, Git top-level, and repository identity against the recorded worktree before any project mutation; it resolves the recovery tmux target to a unique live window id (new helper inbin/backends/tmux.sh) and fails closed — requiring a fresh launch — on missing fields, lost/ambiguous windows, unreadable cwd, or identity mismatch. The stuck-crewmate-recovery skill now requires running this verification after any harness resume.tests/fm-spawn-clone-identity.test.sh(6 regressions covering foreign-pool rejection, clone-identity metadata, historical-cwd resume rejection, lost/ambiguous window fail-closed, and owned-worktree acceptance) and documented the spawn clone-identity boundary inAGENTS.md,docs/architecture.md, anddocs/scripts.md.Risk Assessment
✅ Low: The round-2 commit cleanly resolves the only actionable prior finding by resolving the recorded tmux target to exactly one live stable window id with fail-closed zero/ambiguous handling, adds focused regressions for lost and ambiguous window names, and introduces no new material risk across the re-inspected branch.
Testing
Ran the new clone-identity regression suite plus the surrounding spawn/backend/orca/test-runner suites (all green), then demonstrated the fix end-to-end with a real two-clone shared-pool fixture: fm-spawn.sh refuses the foreign clone's pooled worktree before hooks, launch, or metadata, records the selected clone identity for the owned worktree, and fm-recovery-verify.sh fails closed on a simulated historical-cwd restoration and on lost/ambiguous tmux windows while accepting the recorded worktree; transcript captured as evidence and demo fixtures cleaned up. Shellcheck was not run per testing-stage rules.
Evidence: End-to-end CLI transcript: two-clone shared-pool spawn refusal, clone-identity metadata, and recovery fail-closed/verify scenarios
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 3 issues found → auto-fixed ✅
bin/fm-recovery-verify.sh:78- fm-recovery-verify.sh reads the resumed task's live cwd viatmux display-message -p -tagainst the name-basedwindow=target from meta (fm_backend_target_of_meta returns "$SES:$W", bin/fm-spawn.sh:889). fm-spawn.sh:1085's own comment documents that display-message with a lost/renamed window name falls back to the active client's window — the reason the spawn settle-poll targets the stable window id ($WID) instead. In the verifier the common failure direction is a spurious refusal (fail closed), but a false accept exists: if the task window's name is lost and the active client's pane happens to be inside the recorded worktree (plausible while an operator inspects that worktree during recovery), verification passes while the actual resumed agent runs elsewhere. Harden by resolving the window name to exactly one live window id (failing closed on zero or ambiguous matches) before reading pane_current_path..agents/skills/stuck-crewmate-recovery/SKILL.md:34- The resume-side verification is enforced procedurally: SKILL.md:34 instructs the recovering agent to run bin/fm-recovery-verify.sh after any harness resume, but nothing mechanically gates fm-send instruction delivery on a passed verification. Since resumes are agent-operated, the skill is the available Firstmate-side seam; noting that the fail-closed guarantee for Grok resume still relies on the operating agent following the skill.bin/fm-spawn.sh:301- ORCA_ABORT_METADATA_ALLOWED=0 spans the entire validate_spawn_worktree call, so when an unvalidated Orca worktree cannot be removed during abort, fallback metadata is now skipped for both the new clone-identity failure and the pre-existing isolation failure. The leaked Orca worktree is reported only on stderr and is no longer discoverable from state by later teardown/fleet-sync. This is a deliberate consequence of the no-metadata-before-validation boundary, but operators lose the previous state breadcrumb for the isolation-failure mode.🔧 Fix: resolve recovery tmux target to unique window id, fail closed
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
bash tests/fm-spawn-clone-identity.test.sh— all 6 regression tests pass (foreign-pool rejection, clone-identity metadata, historical-cwd resume rejection, lost/ambiguous window fail-closed, owned-worktree acceptance)bin/fm-test-run.sh tests/fm-spawn-batch.test.sh tests/fm-spawn-dispatch-profile.test.sh tests/fm-spawn-worktree-settle.test.sh tests/fm-backend.test.sh tests/fm-backend-tmux-smoke.test.sh tests/fm-spawn-clone-identity.test.sh— backend-dispatch family, failed=0bin/fm-test-run.sh tests/fm-backend-orca.test.sh tests/fm-test-run.test.sh— orca abort-metadata paths and runner family mapping, failed=0Manual end-to-end CLI demo: built two independent clones of one remote under a shared worktree-pool namespace, ran realbin/fm-spawn.sh(foreign slot refused with no agent launch/metadata/hooks; owned slot succeeds and records project_git_common_dir=) and realbin/fm-recovery-verify.sh(foreign historical cwd refused, lost window name fails closed, recorded worktree verifies)Checkedgit log 593e3a2..b190820commit messages for agent co-author trailers (none) and confirmed Markdown doc changes use one sentence per line with no Treehouse-side edits✅ **Document** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.