fix(axi): isolate AXI suite installs per vessel npm prefix - #36
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Give each Firstmate vessel an isolated AXI npm install prefix derived without configuration from its FM_HOME, eliminating cross-home writes and races from independent update clocks. Use isolated per-vessel prefixes rather than a shared owner or lock, accepting duplicate installs. Preserve the managed tool set and patch/minor auto-update policy; do not uninstall or modify existing shared installs. Reinstall existing versions per vessel as needed, leave shared copies untouched as fallback, make PATH ordering unambiguous so the primary, crew, and secondmate resolve their owning vessel's copy first, cover concurrent two-home updates, and verify the per-home currency clock still works after cutover.
What Changed
FM_HOME($FM_HOME/.local/axi, gitignored) via the newbin/fm-axi-path-lib.sh. Bootstrap, session start, spawn, and dispatch-select prepend that bin directory toPATH,bin/fm-spawn.shtypes the owning vessel's export into each crewmate pane (the secondmate home's own prefix for secondmate launches), and the README launch lines prepend it before the harness starts — so the primary, crew, and secondmate resolve their vessel's copy first and fall back to an inherited external install only before cutover.bin/fm-axi-suite.shseeds each home's prefix on its first currency check — re-installing the already-installed version, or an eligible patch/minor release, without touching or removing any existing shared install — keeping the managed tool set and patch/minor auto-update policy unchanged.state/axi-suite-prefix-v1.cutoverrecords the attempt so a pre-cutover cadence stamp can't postpone it and a partial seed still honours the 24h cadence; a vessel copy that answers but reports no version is discarded and reseeded, one that never answers is reported and left alone, and a locally-ahead build the registry can't supply is reported asAXI_SUITE_REVIEW:instead of a recurring failure. Registry/update work, local version probes (FM_AXI_SUITE_PROBE_TIMEOUT, 15s), and one-time seeding installs (FM_AXI_SUITE_SEED_TIMEOUT, 120s) are now charged to separate budgets, and a seed cut short by an exhausted budget is reported identically from both install paths. Bootstrap'sMISSING:install hints now target the home prefix and invokesetup hooksby tool name so the shared user-global hook config records a portable command.tests/fm-axi-suite.test.shandtests/fm-bootstrap.test.shcover concurrent two-home updates against distinct prefixes, vessel-prefix PATH precedence, the per-home currency clock surviving cutover, unreadable/hung vessel copies, budget-stalled cutovers, and the approved-install path using the vessel prefix. The Document check flagged three remaining prose surfaces (the external admiralty repo's protected-path list, the illustrative gitignore lists inAGENTS.md/CONTRIBUTING.md, and the partialFM_*env-var inventory indocs/configuration.md) as deliberately out of scope.Risk Assessment
✅ Low: Every finding raised across five prior rounds is now resolved with a matching regression test, the three budget accumulators and the repair/fallback path are well bounded and covered, docs and the bootstrap-diagnostics skill are in sync with the emitted tokens, and the only residual - the shared user-global
setup hookswrite that no prefix can isolate - was explicitly accepted by the captain and is documented as a known boundary.Testing
I ran the targeted suites for the changed scripts (axi-suite, bootstrap, spawn dispatch profile, runtime-ignore, gotmp, dispatch-select) — all green — and then built a manual end-to-end demo that drives the real fm-axi-suite/fm-bootstrap/fm-spawn for two vessels sharing one host with a pre-existing global AXI install; the captured transcript shows disjoint per-home npm prefixes under concurrent updates, the shared copy left intact as fallback, vessel-first PATH resolution for primary, crew and secondmate launches, the per-home currency clock still gating then resuming after cutover, and major releases still held for review. This is a shell/CLI change with no rendered UI surface, so the reviewer-visible evidence is the CLI transcript rather than a screenshot. Transient demo workspaces were removed and the worktree is clean.
Evidence: End-to-end transcript: two vessels, isolated AXI prefixes
1. BEFORE CUTOVER $ command -v demo-axi <host>/shared/npm-global/bin/demo-axi (1.0.0) 2. BOTH VESSELS RUN fm-axi-suite.sh --force CONCURRENTLY vessel-alpha: AXI_SUITE_UPDATED: demo-axi 1.0.0 -> 1.0.1 / gh-axi 1.0.0 installed in vessel prefix install -g --prefix <demo>/vessel-alpha/.local/axi demo-axi@1.0.1 install -g --prefix <demo>/vessel-alpha/.local/axi gh-axi@1.0.0 vessel-bravo: AXI_SUITE_UPDATED: demo-axi 1.0.0 -> 1.0.1 / gh-axi 1.0.0 installed in vessel prefix install -g --prefix <demo>/vessel-bravo/.local/axi demo-axi@1.0.1 install -g --prefix <demo>/vessel-bravo/.local/axi gh-axi@1.0.0 3. NO CROSS-HOME WRITES, NO SHARED-INSTALL MUTATION no - every npm install targeted only the calling vessel prefix no - the shared prefix never appears; no uninstall or modify of the shared copy alpha 1.0.1 | bravo 1.0.1 | shared 1.0.0 (untouched fallback) 4. PATH ORDERING PATH="$FM_HOME/.local/axi/bin:$PATH" command -v demo-axi -> <demo>/vessel-alpha/.local/axi/bin/demo-axi (1.0.1) shared bin listed FIRST + fm_axi_prepend_path -> still <demo>/vessel-alpha/.local/axi/bin/demo-axi (1.0.1) 5. CREW AND SECONDMATE LAUNCHES (real fm-spawn.sh, fake pane) crew: typed: export PATH='<demo>/vessel-alpha/.local/axi/bin':$PATH secondmate: typed: export PATH='<demo>/secondmate-home/.local/axi/bin':$PATH 6. PER-HOME CURRENCY CLOCK AFTER CUTOVER state/: axi-suite-prefix-v1.cutover, axi-suite-update.checked cached run inside the window: <silent>, npm calls before=2 after=2 (registry had published 1.0.2) --force: AXI_SUITE_UPDATED: demo-axi 1.0.1 -> 1.0.2 | alpha 1.0.2, bravo 1.0.1, shared 1.0.0 7. POLICY PRESERVED AXI_SUITE_REVIEW: demo-axi major update 1.0.2 -> 2.0.0 (install: npm install -g --prefix <alpha>/.local/axi demo-axi@2.0.0) vessel copy stayed at 1.0.2 8. BOOTSTRAP MISSING: tasks-axi (install: npm install -g --prefix <home>/.local/axi tasks-axi) MISSING: gh-axi (install: npm install -g --prefix <home>/.local/axi gh-axi && PATH=<home>/.local/axi/bin:$PATH gh-axi setup hooks) bin/fm-bootstrap.sh install gh-axi -> installed into <home>/.local/axi/bin, hook setup ranEvidence: Reproduction script for the demo
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 7 issues found → auto-fixed (5) ✅
bin/fm-axi-suite.sh:303- The newstate/axi-suite-prefix-v1.readygate is deleted whenever any suite tool that is on PATH lacks a vessel copy, and the cadence check at line 136 requires that marker. Any tool whose registry lookup ornpm install --prefixcannot succeed (offline home, private/authenticated registry, or a suite entry such asgnhfthat is not installable into this prefix) leaves the marker permanently absent, so the 24h cadence never engages again. Before this change the stamp alone gated the check, so a stuck home still only paid the sweep once a day. Consider recording a separate "cutover attempted" marker so a failed seed still honours the cadence.bin/fm-axi-suite.sh:213- A vessel-prefix install that writes the bin shim but is otherwise broken (interrupted install, node/ABI mismatch) permanently shadows the intact shared copy: the prepended$FM_HOME/.local/axi/binalways wins for the primary, crew, and secondmate,version_ofreturns empty, the loop emitsinstalled version could not be readandcontinues without ever reinstalling, and the readiness loop at line 303 still sees an executable so it writes PREFIX_READY and caches for 24h. The intent's "shared copies untouched as fallback" is unreachable once a bad vessel copy exists. Consider removing/reinstalling the vessel copy when its version cannot be read.bin/fm-axi-suite.sh:188-setup_home_hooksnow runs$FM_HOME/.local/axi/bin/<tool> setup hooksfrom each vessel's own binary during seeding and after every update. The axi hook entries are not in the tracked.claude/settings.json, sosetup hookswrites user-global harness config; if it records the invoking binary's absolute path, the last home to cut over rewrites hook wiring for every other home, and removing that home breaks the others. Previously all vessels invoked the same shared binary so the result was identical regardless of which one ran it. Worth confirming whatsetup hookswrites before this lands on multi-home installs.bin/fm-axi-suite.sh:252- When the installed copy is ahead of the registrylatesttag, seeding runsnpm install -g --prefix ... <tool>@<installed>. That version may not be published (npm-linked dev build, local build, unpublished prerelease), so the install fails, anAXI_SUITE_STUCK:line is emitted every session, and combined with the readiness gate the home never regains its cadence. The old code left locally-ahead tools entirely alone. Falling back to the registrylatest(or skipping the seed) when the exact version is unavailable would avoid the permanent stuck state.bin/fm-axi-path-lib.sh:81-fm_axi_bin_dirinterpolatesfm_axi_prefixvia command substitution and returns printf's status, so when the prefix cannot be resolved (empty home) it prints/binand exits 0 instead of failing.fm_axi_prepend_path's|| return 1therefore never triggers and/bingets prepended to PATH. All current call sites pass a non-empty home, but the guard infm_axi_prefixis defeated. Capture the prefix into a local and propagate its exit status.bin/fm-axi-suite.sh:230- Thehome_installed -eq 0seed block (check-only report vs.seed_home_tool ... || true) is repeated verbatim three times: in the already-current branch (230-241), the locally-ahead branch (253-260), and the pending-major branch (266-273). Extracting a singleensure_home_copy <tool> <version>helper would remove the duplication and keep the three call sites in sync..agents/skills/bootstrap-diagnostics/SKILL.md:52-seed_home_toolemitsAXI_SUITE_UPDATED: <tool> <version> installed in vessel prefix, but the bootstrap-diagnostics skill still documents the token as onlyAXI_SUITE_UPDATED: <tool> <old> -> <new>(a gated patch/minor self-update). docs/configuration.md was updated for the new seeding behavior; this skill, which is what an agent loads to interpret the digest line, was not.🔧 Fix: harden vessel AXI prefix seeding, repair, and cadence
3 issues (1 warning, 2 infos) still open:
bin/fm-axi-suite.sh:187-home_tool_is_readableexecutes the vessel binary viaversion_of, which runs outsidenet_call/boundedand inherits the script's stdin. The new repair pass at line 259 runs this probe for every tool that has a vessel copy, anddiscard_broken_home_toolruns it again at line 198 — on a binary already suspected of being broken. A half-installed copy that stalls rather than exiting (blocks reading inherited stdin, or a shim whose runtime hangs) wedgesfm-axi-suite.shforever, and with it the locked bootstrap atbin/fm-bootstrap.sh:938and session start.FM_AXI_SUITE_NETWORK_TIMEOUTexplicitly bounds registry, install, and hook work but does not cover this. The pre-existingversion_of "$tool"at line 278 shares the exposure, so this widens an existing class rather than creating it. Adding</dev/nulltoversion_ofis a safe non-functional fix; bounding the probe as well would close it fully, though a timeout would then also classify a very slow but working copy as unreadable and remove it.bin/fm-axi-suite.sh:259- In the steady state (vessel copy present and readable) each suite tool's--versionis now spawned twice per check — once by the repair probe at line 259 and once byversion_of "$tool"at line 278, which resolves to the same vessel binary because$AXI_BINis first on PATH — and three times when the probe reports unreadable, sincediscard_broken_home_toolre-evaluateshome_tool_is_readableat line 198 for a verdict the caller just computed. For the six-tool default suite that is 12 node CLI launches instead of 6. Reading the vessel copy's version once and reusing it for the readable/unreadable decision, the discard decision, andinstalledwould remove both duplications.bin/fm-axi-suite.sh:246-ensure_home_copyreturns 1 both when seeding succeeded and when it failed, so it cannot signal "the vessel now owns a copy". At line 294 that leaveshome_present=0after a successful first seed, and the pending-hook-retry branch at line 298 is therefore skipped for a tool that does have a working copy. There is no behavioral gap today only becauseseed_home_toolrunssetup_home_hooksitself on that path, so the pending stuck line still clears — but the two mechanisms are coupled implicitly and an edit to either silently drops the hook retry. Returning 0/1 for "copy present after this call" (or setting an out-variable) would make the guard at line 298 mean what it reads as.🔧 Fix: bound vessel version probes and dedupe version reads
1 error still open:
bin/fm-axi-suite.sh:109-PROBE_DEADLINE=$((STARTED + PROBE_TIMEOUT))makes the probe budget an absolute wall-clock deadline from script start, so time spent innpm viewandnpm install(which draw on the separate 30s network budget) silently consumes it.remaining_budgetthen returns 0 andprobe_call(:137) returns 124 without executing anything, whichread_versionclassifies astimeout. Walking the default 6-tool first cutover with conservative timings (npm view ~2s, npm install ~6s per tool): tool 1 seeds and verifies, but tool 2's post-install verification probe at :245 lands at ~16.6s wall clock with zero budget, soVERSION_READis empty,install_home_toolreturns 1, andseed_home_toolwritesAXI_SUITE_STUCK: <tool> vessel-prefix installation at <prefix> failedfor an install that actually succeeded; tools 3-6 then fail atread_version "$tool"(:345) and emitinstalled version could not be readbeforecontinue, so they are never seeded. With an unreachable registry the firstnpm viewburns the whole network budget and every later tool holding a vessel copy reportsdid not answer --version within 15s(:316) for a binary that was never run - directly contradicting the comment at :133 ("an unreachable registry cannot starve the probes") and the docs/configuration.md claim that this budget covers "cumulative local version probing". The tests miss it because the stub npm returns instantly and the hung-copy test uses a single tool with no preceding slow work. Track probe time actually spent (or cap each probe at PROBE_TIMEOUT with its own accumulator) instead of deriving it from wall clock since start, and report the granted budget in the :317 message rather than always interpolating $PROBE_TIMEOUT.🔧 Fix: charge probe and network budgets separately
2 issues (1 warning, 1 info) still open:
bin/fm-axi-suite.sh:140- Now that the two budgets are charged separately,FM_AXI_SUITE_NETWORK_TIMEOUT(default 30s, covering registry lookups, installs, and hook setup) becomes the binding constraint on the one-time cutover. Before this branch a normal run made sixnpm viewcalls (~9s); the first run in each home now additionally performs up to sixnpm install -g --prefixcalls viaseed_home_tool. At realistic npm timings (~1.5s per view, ~3-6s per install) the 30s budget is consumed around the fourth tool: that tool's install is killed mid-flight and reportsAXI_SUITE_STUCK: <tool> vessel-prefix installation at <prefix> failed(:307), and the remaining tools getNET_GRANTED=0, sonet_call npm view(:398) returns 124 immediately and each reportsAXI_SUITE_STUCK: <tool> latest version lookup failed. Those lines are copied tostate/axi-suite-update.stuck(:452) and re-emitted byemit_cachedon every invocation for the rest of the cadence window. Nothing is corrupted and the home converges over one or two more windows, but the bootstrap-diagnostics skill instructs an agent to investigateAXI_SUITE_STUCKand relay it via a crewmate, so a healthy home's first cutover likely raises a false alarm. The tests miss it because the stub npm is instantaneous and the one slow-timing test raisesFM_AXI_SUITE_NETWORK_TIMEOUTto 60. Options: raise the default, give the one-time seeding installs their own accumulator the way probes just got one, or emit a softer non-STUCK token when seeding is cut short purely by budget exhaustion.bin/fm-axi-suite.sh:118-NET_GRANTEDis declared at file scope but is only ever assigned and read insidenet_call(:142-145); nothing outside consumes it.PROBE_GRANTEDis genuinely global becauseread_version(:233) and the timeout message (:352) read it after the call returns. MakingNET_GRANTEDalocalinnet_callkeeps the two apart and makes it obvious which grant is part of the contract with callers.🔧 Fix: give first-cutover seeding its own bounded budget
2 warnings still open:
bin/fm-axi-suite.sh:506-seed_home_toolgained aSEED_GRANTED -eq 0branch (:369) that reports a spent seeding budget asvessel-prefix seeding ... was not attempted, but the patch/minor update call site at :505 has no equivalent check and always printsAXI_SUITE_STUCK: <tool> automatic update <old> -> <new> failed (npm prefix ...). When the vessel copy is absent and the external copy is behind the registry, the loop skips the threeensure_home_copybranches (:478, :487, :492) and reaches :505, whereinstall_home_toolcorrectly charges the seed budget because[ -x "$path" ]is false - so on a first cutover of a home whose shared installs are stale, every tool seeds through this path andseed_home_tool's disambiguation is never reached at all. If the 120s seeding budget then runs out mid-cutover, the remaining tools report a broken install rather than an unattempted seed, which is the exact misleading alarm this commit set out to remove, and the bootstrap-diagnostics skill will send an agent to investigate the local install path.test_stalled_first_cutover_alarms_and_stays_boundeduses tools whereinstalled == latest, so it exercises only theseed_home_toolbranch. Apply the sameSEED_GRANTED -eq 0check at :505 (or hoist it intoinstall_home_tool's callers) so both entry points describe the same condition identically.bin/fm-axi-suite.sh:307-announce_seedingwrites to stderr on the stated premise that "stdout is collected and emitted once at the end" so a fresh vessel shows live progress "inside a locked bootstrap where the operator sees only a pause"; docs/configuration.md repeats that rationale. Butbin/fm-session-start.sh:278invokes bootstrap asBOOT_OUT=$("$SCRIPT_DIR/fm-bootstrap.sh" 2>&1), so stderr is captured into the same command substitution as stdout and nothing is printed until bootstrap exits. In the automated session-start path the progress lines are buffered exactly like the diagnostics and appear all at once in the digest's BOOTSTRAP section afterwards, so they cannot distinguish normal installation work from a hang while it is happening - live streaming only works when a captain runsbin/fm-bootstrap.shby hand in a terminal. Deciding how to surface it (write to the controlling terminal, have session-start stream bootstrap rather than capture it, or accept post-hoc reporting and correct the two rationale comments) is a behavior choice worth confirming.🔧 Fix: report spent seeding budget identically from both paths
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
bin/fm-test-run.sh tests/fm-axi-suite.test.sh— 21 behaviors incl. concurrent two-home updates, PATH precedence, cutover currency clock, unreadable/hung vessel copy handling, seeding-budget stall reportingbin/fm-test-run.sh tests/fm-bootstrap.test.sh tests/fm-spawn-dispatch-profile.test.sh tests/fm-runtime-ignore.test.sh tests/fm-gotmp.test.sh tests/fm-dispatch-select.test.shManual end-to-end demo driving the realbin/fm-axi-suite.sh,bin/fm-bootstrap.shandbin/fm-spawn.shfor two vessels against a fake npm registry and fake tmux pane:FM_DEMO_REPO=$PWD bash demo-vessel-axi-isolation.shConcurrent cutover of vessel-alpha and vessel-bravo, then inspection of each vessel's npm call log, resulting binaries, and the untouched sharednpm-globalcopyPATH resolution checks: README primary launch line,fm_axi_prepend_pathwith the shared bin listed first, plus the literalexport PATH=...line fm-spawn types for a crewmate and for a secondmateCurrency-clock check after cutover: silent cached run inside the window with zero npm traffic, then--forcepicking up 1.0.1 -> 1.0.2 for alpha onlyPolicy check: registry offering 2.0.0 yieldsAXI_SUITE_REVIEW:with the vessel-prefix install hint and leaves the vessel copy at 1.0.2bin/fm-bootstrap.shMISSING disclosure andbin/fm-bootstrap.sh install gh-axion a fresh home, verifying the npm call, the installed binary location, and the hook-setup invocationgit show ce8309f:bin/fm-axi-suite.sh | grep '^SUITE='vs current, andgrep 'uninstall|rm -f|rm -rf' bin/fm-axi-suite.shto confirm the tool set and that no shared install is removeddocs/admiralty-fleet-repo.md:63- docs/admiralty-fleet-repo.md:63 states that the external fleet repo'sfleet/bin/fmf-vendor-bump.shtreatsdata/,state/,config/,projects/,.no-mistakes/,graphify-out/, and.envas structurally out of reach. That is still an accurate report of that script, but an admiralty-deployed vessel now also creates the home-private.local/axi/prefix, which the protected set does not cover. The fix belongs in the admiralty repo's script, not here, so I did not edit the sentence describing it.AGENTS.md:53- AGENTS.md:53 and CONTRIBUTING.md:41 both enumerate captain-private gitignored paths (.env,data/,state/,config/,projects/,.no-mistakes/,graphify-out/) and now omit.local/axi/. I deliberately left both alone: they already omitted.lavish/before this change, so they read as illustrative rather than exhaustive, and the tracked root.gitignore(updated by this change) is the enforcing owner. Adding the entry would mean maintaining a third and fourth prose copy of that list.docs/configuration.md:556- The## Environment variablesblock in docs/configuration.md lists noFM_AXI_SUITE_*variable at all, so the two new ones (FM_AXI_SUITE_PROBE_TIMEOUT,FM_AXI_SUITE_SEED_TIMEOUT) join three pre-existing absentees rather than creating fresh drift. The block covers roughly 122 of ~400FM_*variables, and the prose owner (docs/configuration.md "AXI-suite self-update") does document all five, so I left it. A follow-up could decide whether that block is meant to be an inventory and reconcile it in one pass.✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.