Skip to content

feat(bin): add a repeatable model-panel capability - #33

Merged
Freudator86 merged 8 commits into
mainfrom
fm/fm-model-panel-capability
Jul 27, 2026
Merged

feat(bin): add a repeatable model-panel capability#33
Freudator86 merged 8 commits into
mainfrom
fm/fm-model-panel-capability

Conversation

@Freudator86

Copy link
Copy Markdown
Owner

Intent

Make the captain's three-model panel a first-class, repeatable firstmate capability instead of something firstmate hand-assembles each time. A panel is two analysts answering the same question independently on different models, neither able to see the other's work, plus a third model that judges both reports. Two hand-runs on 2026-07-27 proved it pays: the analysts disagreed on facts (not style), the judge caught both making the same mistake for the same reason (a network test reproduced from the wrong vantage point), and the formation surfaced seven captain decisions that existed only in prose. The value is independence plus adversarial verification, not three opinions.

Deliverable: one documented entry point (bin/fm-model-panel.sh) plus a thin captain-invocable skill surface (.agents/skills/panel), with roles pinned in code, models configurable, the single-model degradation case handled explicitly with a test, coverage that analysts actually dispatch on different models when configured so, and documentation of when a panel is worth its cost.

Design decisions made deliberately, so they are choices rather than oversights:

  1. NO MODEL NAMES IN TRACKED CODE. This was the captain's explicit design call. Roles (analyst_a, analyst_b, judge) are pinned; which model fills each comes from local gitignored config/model-panel.json. A tracked script naming today's models would rot silently, exactly like the stale backlog note that triggered this session, and config is also what lets each home run a panel on the models it can actually reach.

  2. REUSE, DO NOT REIMPLEMENT. Every panel member is an ordinary scout: the brief comes from bin/fm-brief.sh --scout with its {TASK} placeholder replaced, dispatch goes through bin/fm-spawn.sh, and every role profile resolves through bin/fm-dispatch-select.sh so panel profiles inherit its validation and quota-aware array selection. There is deliberately no second selector, no second lifecycle, and no new brief scaffold.

  3. STAGED, NOT BLOCKING. start dispatches the analysts concurrently; advance creates the judge only once every analyst report exists. A single blocking command would fight firstmate's supervision model, where scouts report done and wake the supervisor. advance is idempotent, prints a waiting: line when a report is missing, and every non-terminal outcome names the next command, so one entry point still carries a panel from question to answer.

  4. HONEST DEGRADATION, BY EXPLICIT CAPTAIN INSTRUCTION. Two identical analysts are not a panel, and a panel that quietly is not one is worse than no panel. When both analysts resolve to one model identity, start refuses with exit 4 and names both the configuration fix and the opt-in --reduced form. Model identity strips the provider prefix and any :suffix, so claude/claude-opus-5 and pi/anthropic/claude-opus-5:1m are correctly ONE model. The reduced form is a named single-analyst review, labelled as such in the briefs, the panel record, and the judge's own report, and is never selected implicitly. When no third distinct model exists the judge may share an analyst's model: that warns and proceeds, because the judge's independence comes from re-verifying against live state with both reports in hand.

  5. DOCUMENTED DEFAULT WITHOUT PINNING MODELS. With no config/model-panel.json, each role falls back to config/crew-dispatch.json's top-level default profile set, so a home that already declares its runtimes gets a panel for free. With neither file, the panel refuses and names both files rather than guessing.

  6. NOT INHERITED BY SECONDMATE HOMES. config/model-panel.json is deliberately absent from FM_INHERITABLE_CONFIG, same reasoning as config/backend: it names the models one specific home can reach, and force-pushing the primary's list downstream would overwrite exactly the local knowledge that lets each home field a real panel.

  7. BLIND JUDGING. The judge is given both reports labelled A and B, and is deliberately NOT told which model wrote which, to keep model prestige out of the verdict. A test asserts the model names are absent from the judge brief.

  8. EFFORT IS NOT INJECTED. Panels are ambiguous investigation work, so xhigh is usually right, but the script never injects a default effort: dispatch-select rejects an effort value a harness does not accept (opencode accepts none), so injecting one would break valid configurations. The recommendation lives in docs and the example config instead.

  9. THE BRIEF TEXT IS LOAD-BEARING, NOT DECORATION. Both hand-runs were told a concrete example of a stale record misleading firstmate and both found further stale records because of it, so the generated briefs carry that verify-status-prose-never-trust-it discipline with the incident named, plus a vantage-point check (the shared mistake the judge caught), a highest-conviction-calls section the judge scores, and an unresolved-captain-decisions inventory. The scout scaffold's decision-hold completion gate carries through to every member unchanged.

Scope the captain set: build the capability only. Do not run a panel as part of this task, and do not change existing dispatch behaviour for ordinary tasks. Nothing in fm-spawn, fm-brief, or fm-dispatch-select was modified; the only edits to existing files are additive registration (AGENTS.md config-map line, data-map line, and one skill trigger; docs/configuration.md's new owner section; .gitignore; README and docs/scripts.md rows; the gate-refusal entrypoint list in docs/architecture.md; and test-family registration in bin/fm-test-run.sh).

Firstmate repo conventions were followed deliberately: one sentence per line in tracked markdown, plain dashes, no agent co-author, shellcheck-clean via bin/fm-lint.sh, a colocated tests/fm-model-panel.test.sh registered in the pure-contract-unit family, and the one-owner rule (script header owns mechanics, docs/configuration.md owns the config schema and degradation contract, the panel skill owns when a panel is worth its cost). Tests are hermetic: they copy bin/, stub only fm-spawn.sh to record its argv, and point FM_DISPATCH_QUOTA_AXI at an absent command so no test reaches a live quota service.

What Changed

  • Adds bin/fm-model-panel.sh with start/advance/status: two analysts are dispatched concurrently on different models to answer one question independently, and a third model is dispatched only once every analyst has finished, judging both reports blind (labelled A and B, model names deliberately absent from the judge brief). Every member is an ordinary scout - the brief comes from bin/fm-brief.sh --scout, dispatch goes through bin/fm-spawn.sh, and each role profile resolves through bin/fm-dispatch-select.sh, so there is no second selector, lifecycle, or brief scaffold. Roles are pinned in code; the models filling them come from a local gitignored config/model-panel.json, falling back to config/crew-dispatch.json's default profile set, and refusing while naming both files when neither supplies a role.
  • Makes the failure and degradation paths explicit rather than silent: start exits 4 when both analysts resolve to one model identity (provider prefix and :suffix stripped) and points at the opt-in --reduced single-analyst review; advance gates every member on a terminal done:/failed: status event and a non-empty report, latching observed readiness into the panel record, and distinguishes waiting:, wedged: (report written, member gone unsignalled - waivable per-member via --accept-unfinished), stood down: (analyst finished with no report), and verdict lost: (judge finished with no report - replaceable via --rejudge). bin/fm-classify-lib.sh gains status_is_finished_verb/status_has_finished_event to back that gate, and the panel now sources bin/fm-gate-refuse-lib.sh so it exits 3 inside a no-mistakes gate worktree.
  • Registers the capability across the existing surfaces: a .agents/skills/panel/SKILL.md captain-invocable skill owning when a panel is worth its cost, a docs/configuration.md "Model panel roles" section owning the schema and degradation contract (with docs/examples/model-panel.json), plus AGENTS.md config/data-map and skill-trigger lines, README and docs/scripts.md rows, .gitignore, the gate-refusal entrypoint list in docs/architecture.md, and test-family registration in bin/fm-test-run.sh. config/model-panel.json is deliberately excluded from the inheritable config set. Hermetic coverage lands in tests/fm-model-panel.test.sh (23 cases: distinct-model dispatch, exit-4 refusal, reduced form, blind judge brief, gate states) plus a new panel section in tests/fm-gate-refuse.test.sh; both suites pass. No existing dispatch behavior for ordinary tasks was changed.

Risk Assessment

✅ Low: The change closes the last classification gap by reusing outcomes and machinery that already exist, adds no new state or operator action, is covered by three targeted tests including an early-window guard, and leaves only unreachable message code behind.

Testing

I ran the change's colocated test file and the modified gate-refusal test (all passing, and the runner's coverage guard confirms the new family registration), then went beyond unit assertions by driving the real fm-model-panel.sh end to end in a hermetic sandbox home with only fm-spawn.sh stubbed, capturing three CLI transcripts plus the actual generated analyst and judge briefs: a full panel from question to verdict showing the analysts dispatched on two different models and the judge created only after both analysts wrote a terminal event and a non-empty report, the blind judge brief with no model or harness names in it, the exit-4 refusal when both analysts resolve to one model identity across different harnesses together with the opt-in --reduced form labelled a single-analyst review everywhere, and the documented config fallback and refusal paths. This change has no UI, HTML, or rendered surface - it is a shell entry point plus a skill - so CLI transcripts and the generated brief files are the end-user-visible artifacts and no screenshot applies. I did not run a live panel (the intent forbids it) and did not run the full suite; the sandbox was removed and the worktree is clean.

Evidence: Full panel lifecycle transcript: dry-run, start on two different models, staged judge gate, blind judge brief, complete, panel record
==============================================================
 1. FULL PANEL LIFECYCLE - question to verdict
==============================================================
config/model-panel.json in this home:
{"roles":{
  "analyst_a":{"harness":"claude","model":"claude-opus-5","effort":"xhigh"},
  "analyst_b":{"harness":"codex","model":"gpt-5.6-sol","effort":"xhigh"},
  "judge":{"harness":"grok","model":"grok-4-fast","effort":"high"}}}

--- the question the captain wants answered (question.md)
Does the backlog note claiming crew dispatch ignores per-role effort still hold?
Verify against live state, not against what the record says.

--- show the captain who will argue before spending anything

$ bin/fm-model-panel.sh start --id q1 --project /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/subject --question-file /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/question.md --dry-run
panel: q1 form=panel project=/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/subject
panel: analyst-a {"harness":"claude","model":"claude-opus-5","effort":"xhigh"}
panel: analyst-b {"harness":"codex","model":"gpt-5.6-sol","effort":"xhigh"}
panel: judge {"harness":"grok","model":"grok-4-fast","effort":"high"}
[exit 0]

--- nothing was created by the dry run
(no dispatches)

--- start the panel for real

$ bin/fm-model-panel.sh start --id q1 --project /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/subject --question-file /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/question.md
panel: q1 form=panel project=/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/subject
dispatched: q1-a harness=claude model=claude-opus-5 effort=xhigh
dispatched: q1-b harness=codex model=gpt-5.6-sol effort=xhigh
panel: next step is `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/bin/fm-model-panel.sh advance q1` once every analyst reports done
[exit 0]

--- what the panel actually asked the fleet to dispatch (stubbed fm-spawn argv)
q1-a /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/subject --harness claude --scout --model claude-opus-5 --effort xhigh
q1-b /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/subject --harness codex --scout --model gpt-5.6-sol --effort xhigh

--- the two analysts are on DIFFERENT models, and the judge is not dispatched yet
--model claude-opus-5
--model gpt-5.6-sol

--- both analysts received the identical question text
10:Does the backlog note claiming crew dispatch ignores per-role effort still hold?
10:Does the backlog note claiming crew dispatch ignores per-role effort still hold?

--- the ONLY differences between the two analyst briefs are their own task paths
4c4
< You are analyst A on a firstmate model panel.
---
> You are analyst B on a firstmate model panel.
16c16
< Do not read their report at `$HOME/data/q1-b/report.md`, and do not read their worktree, pane, status file, or brief.
---
> Do not read their report at `$HOME/data/q1-a/report.md`, and do not read their worktree, pane, status file, or brief.
37c37
< Write the report to `$HOME/data/q1-a/report.md` as the scout definition of done below requires, and end it with these two sections.
---
> Write the report to `$HOME/data/q1-b/report.md` as the scout definition of done below requires, and end it with these two sections.
66c66
<    `echo "{state}: {one short line}" >> '$HOME/state/q1-a.status'`
---
>    `echo "{state}: {one short line}" >> '$HOME/state/q1-b.status'`
84c84
< Write your findings to `$HOME/data/q1-a/report.md`.
---
> Write your findings to `$HOME/data/q1-b/report.md`.

--- the load-bearing brief discipline both analysts carry
20:## Verify status prose, never trust it
29:## Check your vantage point
39:### Highest-conviction calls
45:### Unresolved captain decisions
86:Before reporting done, read and follow `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/.agents/skills/decision-hold-lifecycle/SKILL.md` and pass its shared completion gate for the report and any visual review.
16:Do not read their report at `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/data/q1-b/report.md`, and do not read their worktree, pane, status file, or brief.
16:Do not read their report at `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/data/q1-a/report.md`, and do not read their worktree, pane, status file, or brief.

--- advance before anyone has reported: idempotent wait, judge stays undispatched

$ bin/fm-model-panel.sh advance q1
waiting: the judge is created only once every analyst has finished AND left a non-empty report; q1-a has written no terminal status event yet (no done: or failed: line in /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/state/q1-a.status) and no report; q1-b has written no terminal status event yet (no done: or failed: line in /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/state/q1-b.status) and no report
panel: next step is `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/bin/fm-model-panel.sh advance q1` after the next analyst finishes
[exit 0]

--- analyst A writes its report and declares itself finished

$ bin/fm-model-panel.sh advance q1
waiting: the judge is created only once every analyst has finished AND left a non-empty report; q1-b has written no terminal status event yet (no done: or failed: line in /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/state/q1-b.status) and no report
panel: next step is `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/bin/fm-model-panel.sh advance q1` after the next analyst finishes
[exit 0]

--- analyst B writes a report but has NOT declared itself finished yet

$ bin/fm-model-panel.sh advance q1
waiting: the judge is created only once every analyst has finished AND left a non-empty report; q1-b left a report but has not written its terminal status event yet
panel: next step is `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/bin/fm-model-panel.sh advance q1` after the next analyst finishes
[exit 0]

--- a report that exists is not a report that is finished - judge still not dispatched
0

--- analyst B declares itself finished; now the judge is created

$ bin/fm-model-panel.sh advance q1
dispatched: q1-judge harness=grok model=grok-4-fast effort=high
panel: next step is `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/bin/fm-model-panel.sh advance q1` once the judge reports done
[exit 0]

--- the judge dispatch, on its own configured model
q1-judge /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/subject --harness grok --scout --model grok-4-fast --effort high

--- BLIND JUDGING: the judge gets both reports labelled A and B...
16:- Analyst A: `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/data/q1-a/report.md`
17:- Analyst B: `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/data/q1-b/report.md`
45:Write the report to `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/data/q1-judge/report.md` as the scout definition of done below requires, covering these sections in order.
90:Write your findings to `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/data/q1-judge/report.md`.

--- ...and is NOT told which model wrote which
judge brief does not mention claude-opus-5: ok
judge brief does not mention gpt-5.6-sol: ok
judge brief does not mention claude: ok
judge brief does not mention codex: ok

--- the judge writes a verdict but has not signalled done

$ bin/fm-model-panel.sh advance q1
waiting: the panel is complete only once the judge has finished AND left a non-empty report; q1-judge left a report but has not written its terminal status event yet
panel: next step is `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/bin/fm-model-panel.sh advance q1` once the judge reports done
[exit 0]

--- the judge declares itself finished

$ bin/fm-model-panel.sh advance q1
complete: /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/data/q1-judge/report.md
panel: read that report, relay its findings, and pass the shared completion gate before standing the panel down
[exit 0]

--- the durable panel record (data/q1/panel.meta)
panel=q1
form=panel
project=/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/subject
question=/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/data/q1/question.md
created=2026-07-27T17:23:29Z
stage=complete
analyst_a_task=q1-a
analyst_a_profile={"harness":"claude","model":"claude-opus-5","effort":"xhigh"}
analyst_b_task=q1-b
analyst_b_profile={"harness":"codex","model":"gpt-5.6-sol","effort":"xhigh"}
judge_task=q1-judge
judge_profile={"harness":"grok","model":"grok-4-fast","effort":"high"}
ready_members=q1-a q1-b q1-judge

--- the verdict the operator relays
Verdict: both analysts reproduced the check from the wrong vantage point.

--- status is read-only and repeatable

$ bin/fm-model-panel.sh status q1
panel=q1
form=panel
project=/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/subject
question=/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/full/data/q1/question.md
created=2026-07-27T17:23:29Z
stage=complete
analyst_a_task=q1-a
analyst_a_profile={"harness":"claude","model":"claude-opus-5","effort":"xhigh"}
analyst_b_task=q1-b
analyst_b_profile={"harness":"codex","model":"gpt-5.6-sol","effort":"xhigh"}
judge_task=q1-judge
judge_profile={"harness":"grok","model":"grok-4-fast","effort":"high"}
ready_members=q1-a q1-b q1-judge
[exit 0]
Evidence: Honest degradation transcript: exit-4 refusal on one model behind two harnesses, then the labelled --reduced single-analyst review
==============================================================
 2. HONEST DEGRADATION - one model behind two harnesses
==============================================================
config/model-panel.json in this home (same model, two harnesses,
one carrying a provider prefix and a :suffix):
{"roles":{
  "analyst_a":{"harness":"claude","model":"claude-opus-5"},
  "analyst_b":{"harness":"pi","model":"anthropic/claude-opus-5:1m"},
  "judge":{"harness":"claude","model":"claude-opus-5"}}}

--- start refuses: this would not be a panel

$ bin/fm-model-panel.sh start --id d1 --project /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/degrade/subject Which ordering is right?
error: this would not be a panel: both analysts resolve to the model claude-opus-5.
  Two identical analysts are not independent, and presenting them as a panel is worse than running none.
  Configure a second distinct model as roles.analyst_b in /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/degrade/config/model-panel.json (see docs/examples/model-panel.json),
  or re-run with --reduced for the single-analyst review, which is labelled as such everywhere it appears.
[exit 4]

--- nothing was dispatched, nothing was written
(no dispatches)

--- the opt-in reduced form, which is never selected implicitly

$ bin/fm-model-panel.sh start --id d1 --project /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/degrade/subject --reduced Which ordering is right?
panel: no third distinct model is available, so the judge runs on claude-opus-5, the same model as one analyst; it still re-verifies against live state with every report in hand
panel: d1 form=single-analyst-review project=/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/degrade/subject
dispatched: d1-a harness=claude model=claude-opus-5 effort=default
panel: next step is `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/bin/fm-model-panel.sh advance d1` once every analyst reports done
[exit 0]

--- one analyst plus a judge; no analyst B
d1-a /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/degrade/subject --harness claude --scout --model claude-opus-5

--- labelled a single-analyst review in the panel record
2:form=single-analyst-review

--- labelled in the analyst's own brief
4:You are the analyst on a firstmate single-analyst review.

--- and the judge is told to say so in its report

$ bin/fm-model-panel.sh advance d1
dispatched: d1-judge harness=claude model=claude-opus-5 effort=default
panel: next step is `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/bin/fm-model-panel.sh advance d1` once the judge reports done
[exit 0]
4:You are the judge on a firstmate single-analyst review, the reduced form of the model panel.
7:State in your report's first line that this was a single-analyst review rather than a panel, so its output is never presented as an independent panel result.
Evidence: Configuration transcript: no-config refusal naming both files, crew-dispatch default fallback, non-inheritance, gate refusal
==============================================================
 3. CONFIGURATION - documented fallback and honest refusal
==============================================================

--- no config/model-panel.json and no config/crew-dispatch.json

$ bin/fm-model-panel.sh start --id n1 --project /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/noconfig/subject Anything?
error: no profile for panel role 'analyst_a': add roles.analyst_a to /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/noconfig/config/model-panel.json (see docs/examples/model-panel.json) or a default profile set in /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/noconfig/config/crew-dispatch.json
error: panel role analyst_a could not be resolved:
error: dispatch input must be a rule, profile, or profile array
[exit 1]

--- no config/model-panel.json, but crew-dispatch.json declares a default:
{"default":{"harness":"claude","model":"claude-opus-5"}}

--- every role falls back to that default set, so the single-model home

--- still gets the honest refusal rather than a fake panel

$ bin/fm-model-panel.sh start --id c1 --project /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/crewdefault/subject Anything?
error: this would not be a panel: both analysts resolve to the model claude-opus-5.
  Two identical analysts are not independent, and presenting them as a panel is worse than running none.
  Configure a second distinct model as roles.analyst_b in /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/crewdefault/config/model-panel.json (see docs/examples/model-panel.json),
  or re-run with --reduced for the single-analyst review, which is labelled as such everywhere it appears.
[exit 4]

--- and the reduced form works out of the box on that same fallback

$ bin/fm-model-panel.sh start --id c1 --project /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/crewdefault/subject --reduced Anything?
panel: no third distinct model is available, so the judge runs on claude-opus-5, the same model as one analyst; it still re-verifies against live state with every report in hand
panel: c1 form=single-analyst-review project=/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/crewdefault/subject
dispatched: c1-a harness=claude model=claude-opus-5 effort=default
panel: next step is `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/bin/fm-model-panel.sh advance c1` once every analyst reports done
[exit 0]
c1-a /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/crewdefault/subject --harness claude --scout --model claude-opus-5

--- the panel entry point is registered in the gate refusal, so a no-mistakes

--- gate agent cannot drive it (run with the test bypass removed)

$ bin/fm-model-panel.sh start --id g1 --project /home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/gatecheck/subject Anything?
error: no-mistakes gate agent must not drive the fleet (NO_MISTAKES_GATE set)
[exit 3]

--- config/model-panel.json is deliberately NOT inherited by secondmate homes
29:# Extensible by design: FM_INHERITABLE_CONFIG is the single declared list of
49:FM_INHERITABLE_CONFIG="${FM_INHERITABLE_CONFIG:-crew-dispatch.json crew-harness backlog-backend herdr-presentation-spaces firstmate-update-base}"
402:  for item in $FM_INHERITABLE_CONFIG; do
0
Evidence: Generated judge brief an actual judge model would receive (reports labelled A and B, models deliberately withheld)
You are a crewmate: an autonomous worker agent managed by firstmate. Work on your own; do not wait for a human.

# Task
You are the judge on a firstmate model panel.
Two analysts answered the same question independently, on different models, with no access to each other's work.
Your job is not to referee their rhetoric, split the difference, or pick a winner wholesale.
It is to re-verify what they claim against live state and produce the answer the captain should act on.

## The question every analyst was given

Does the backlog note claiming crew dispatch ignores per-role effort still hold?
Verify against live state, not against what the record says.

## The reports

- Analyst A: `$HOME/data/q1-a/report.md`
- Analyst B: `$HOME/data/q1-b/report.md`

They are labelled A and B deliberately, and the models behind them are deliberately withheld from you.
Judge the work, not the runtime.

## Re-verify, do not referee

For every load-bearing claim the answer rests on, check it yourself against the live source before you accept it.
Agreement is not evidence: the most valuable thing this formation produces is the mistake made for the same reason on both sides, which neither could have caught alone.
Look specifically for a shared assumption, a shared wrong vantage point, and shared reliance on the same stale written record.

## Verify status prose, never trust it

Written state in this fleet goes stale silently, and a stale record that still reads as current is the most reliable way to be confidently wrong here.
Backlog notes, task notes, status lines, earlier reports, code comments, and documentation are all claims about some past moment, not the current truth.
This is not a hypothetical: a hand-run of this panel was handed a backlog note describing work as still in flight long after it had landed, and only caught it by checking.
So for every load-bearing claim in your report, verify it yourself against the live source - the actual command output, the actual file at the actual path, the actual process or API state - and cite that evidence with the command you ran or the `file:line` you read.
Where prose and live state disagree, live state wins, and the disagreement is itself a finding worth reporting.
Where you could not verify something, say so explicitly instead of inheriting the claim.

## Check your vantage point

Confirm where you are actually running before concluding anything about the environment.
You are in a disposable worktree, which may sit on a different machine, container, network position, or account than the subject of the question.
A test run from the wrong vantage point returns a real result about the wrong system, and both analysts in one hand-run made exactly that mistake together.

## Report contract

Write the report to `$HOME/data/q1-judge/report.md` as the scout definition of done below requires, covering these sections in order.

1. The answer to the question, stated so the captain can act on it.
2. Every point where the reports disagreed on fact, and what the live evidence you checked actually shows.
3. Shared mistakes: anything wrong on both sides for the same reason, and why it was missed.
4. Scored highest-conviction calls: mark each one correct, partly correct, or wrong, name the evidence you checked it against, and say which analysis was stronger on which specific question.
   "Both were fine" is acceptable only where you verified it.
5. What remains unverified or unknown, stated plainly rather than smoothed over.
6. Unresolved captain decisions surfaced by either report or by your own verification, each with its options.
   An empty list is a valid answer; a missing list is not.

# Herdr lifecycle declaration - NOT ENABLED
**HARD SAFETY GATE:** this scaffold cannot inspect the task text that replaces `{TASK}` later.
If the task will start, stop, delete, restart, profile, or otherwise drive Herdr lifecycle behavior, stop and regenerate the brief with `--herdr-lab` before dispatch.
Do not add Herdr lifecycle commands to this unguarded brief by hand.

# Setup
You are in a disposable git worktree of subject, at a detached HEAD on a clean default branch.
This is a SCOUT task: the deliverable is a written report, not a PR.
The worktree is your laboratory - install, run, edit, and make scratch commits freely; all of it is discarded at teardown.
The report is the only thing that survives, so anything worth keeping must be in it.

# Rules
1. Never push to any remote and never open a PR.
2. Stay inside this worktree; the only files you may write outside it are the report and the status file below.
3. Use gh-axi for GitHub operations and chrome-devtools-axi for browser operations.
4. Report status by appending one line:
   `echo "{state}: {one short line}" >> '$HOME/state/q1-judge.status'`
   States: working, needs-decision, blocked, paused, done, failed.
   Each append wakes firstmate, so report sparingly: only phase changes a supervisor
   would act on and the needs-decision/blocked/paused/done/failed states. No step-by-step
   FYI progress lines; firstmate reads your pane for that.
   Use `paused: {why}` - distinct from `blocked:` - ONLY when you are deliberately idling on a
   known external wait you expect to clear on its own (an upstream release, a rate-limit reset):
   firstmate then leaves your idle pane alone and rechecks it on a long cadence instead of
   treating it as a possible wedge. Use `blocked:` when you are stuck and need help.
5. If you hit the same obstacle twice, append `blocked: {why}` and stop; firstmate will help.
6. If a decision belongs to a human (product choices, destructive actions),
   append `needs-decision: {summary of options}` and stop. Firstmate will reply with the decision.
   When firstmate replies or a blocker clears and you resume, append `resolved: {how it was decided or unblocked}` (add the same `[key=<slug>]` if you opened it with one) so the decision or blocker is durably closed and does not keep resurfacing.
7. Never stop, restart, or update the shared `no-mistakes` daemon - it is one instance serving
   every lane/home, so restarting it kills other lanes' in-flight pipeline runs. On ANY no-mistakes
   daemon error, append `blocked: {the daemon error}` and stop; only firstmate manages the daemon.

# Definition of done
Write your findings to `$HOME/data/q1-judge/report.md`.
The report must stand alone: what you did, what you found, the evidence (commands run, output, file:line references), and what you recommend.
Before reporting done, read and follow `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/.agents/skills/decision-hold-lifecycle/SKILL.md` and pass its shared completion gate for the report and any visual review.
When the report is complete, append `done: {one-line conclusion}` to the status file and stop.
If your findings reveal work that should ship (e.g. you reproduced a bug and the fix is clear), say so in the report; firstmate may promote this task in place, and you would then receive mode-specific ship instructions as a follow-up message.
Evidence: Generated analyst A brief (question, independence rule, verify-status-prose and vantage-point discipline, decisions inventory)
You are a crewmate: an autonomous worker agent managed by firstmate. Work on your own; do not wait for a human.

# Task
You are analyst A on a firstmate model panel.
Two analysts answer the same question independently, on different models, and a third model then judges both answers and re-verifies their claims.
Your report is one half of that input, and its whole value is that it was reached independently.

## The question

Does the backlog note claiming crew dispatch ignores per-role effort still hold?
Verify against live state, not against what the record says.

## Independence - hard rule

Another analyst is answering this exact question right now, on a different model, in a different worktree.
Do not read their report at `$HOME/data/q1-b/report.md`, and do not read their worktree, pane, status file, or brief.
Do not coordinate with them in any way; if you encounter their work incidentally, stop reading it and say so in your report.
Reaching the same conclusion independently is a strong signal, and copying it destroys the only thing this formation buys.

## Verify status prose, never trust it

Written state in this fleet goes stale silently, and a stale record that still reads as current is the most reliable way to be confidently wrong here.
Backlog notes, task notes, status lines, earlier reports, code comments, and documentation are all claims about some past moment, not the current truth.
This is not a hypothetical: a hand-run of this panel was handed a backlog note describing work as still in flight long after it had landed, and only caught it by checking.
So for every load-bearing claim in your report, verify it yourself against the live source - the actual command output, the actual file at the actual path, the actual process or API state - and cite that evidence with the command you ran or the `file:line` you read.
Where prose and live state disagree, live state wins, and the disagreement is itself a finding worth reporting.
Where you could not verify something, say so explicitly instead of inheriting the claim.

## Check your vantage point

Confirm where you are actually running before concluding anything about the environment.
You are in a disposable worktree, which may sit on a different machine, container, network position, or account than the subject of the question.
A test run from the wrong vantage point returns a real result about the wrong system, and both analysts in one hand-run made exactly that mistake together.

## Report contract

Write the report to `$HOME/data/q1-a/report.md` as the scout definition of done below requires, and end it with these two sections.

### Highest-conviction calls

Three to five claims you are most confident in, most important first.
Each one gets the claim in one sentence, the evidence you verified it against, and what changes if you are wrong.
These are scored individually against the other work in this formation, so put your real convictions here rather than the safe ones.

### Unresolved captain decisions

Every choice your investigation surfaced that belongs to the captain rather than to you, each with its options.
An empty list is a valid answer; a missing list is not.

# Herdr lifecycle declaration - NOT ENABLED
**HARD SAFETY GATE:** this scaffold cannot inspect the task text that replaces `{TASK}` later.
If the task will start, stop, delete, restart, profile, or otherwise drive Herdr lifecycle behavior, stop and regenerate the brief with `--herdr-lab` before dispatch.
Do not add Herdr lifecycle commands to this unguarded brief by hand.

# Setup
You are in a disposable git worktree of subject, at a detached HEAD on a clean default branch.
This is a SCOUT task: the deliverable is a written report, not a PR.
The worktree is your laboratory - install, run, edit, and make scratch commits freely; all of it is discarded at teardown.
The report is the only thing that survives, so anything worth keeping must be in it.

# Rules
1. Never push to any remote and never open a PR.
2. Stay inside this worktree; the only files you may write outside it are the report and the status file below.
3. Use gh-axi for GitHub operations and chrome-devtools-axi for browser operations.
4. Report status by appending one line:
   `echo "{state}: {one short line}" >> '$HOME/state/q1-a.status'`
   States: working, needs-decision, blocked, paused, done, failed.
   Each append wakes firstmate, so report sparingly: only phase changes a supervisor
   would act on and the needs-decision/blocked/paused/done/failed states. No step-by-step
   FYI progress lines; firstmate reads your pane for that.
   Use `paused: {why}` - distinct from `blocked:` - ONLY when you are deliberately idling on a
   known external wait you expect to clear on its own (an upstream release, a rate-limit reset):
   firstmate then leaves your idle pane alone and rechecks it on a long cadence instead of
   treating it as a possible wedge. Use `blocked:` when you are stuck and need help.
5. If you hit the same obstacle twice, append `blocked: {why}` and stop; firstmate will help.
6. If a decision belongs to a human (product choices, destructive actions),
   append `needs-decision: {summary of options}` and stop. Firstmate will reply with the decision.
   When firstmate replies or a blocker clears and you resume, append `resolved: {how it was decided or unblocked}` (add the same `[key=<slug>]` if you opened it with one) so the decision or blocker is durably closed and does not keep resurfacing.
7. Never stop, restart, or update the shared `no-mistakes` daemon - it is one instance serving
   every lane/home, so restarting it kills other lanes' in-flight pipeline runs. On ANY no-mistakes
   daemon error, append `blocked: {the daemon error}` and stop; only firstmate manages the daemon.

# Definition of done
Write your findings to `$HOME/data/q1-a/report.md`.
The report must stand alone: what you did, what you found, the evidence (commands run, output, file:line references), and what you recommend.
Before reporting done, read and follow `/home/coditan/.cache/no-mistakes-tmp/fm-panel-demo.VtMvPK/root/.agents/skills/decision-hold-lifecycle/SKILL.md` and pass its shared completion gate for the report and any visual review.
When the report is complete, append `done: {one-line conclusion}` to the status file and stop.
If your findings reveal work that should ship (e.g. you reproduced a bug and the fix is clear), say so in the report; firstmate may promote this task in place, and you would then receive mode-specific ship instructions as a follow-up message.
- Evidence: Driver used to produce the transcripts (hermetic sandbox, only fm-spawn.sh stubbed) (local file: /home/coditan/.cache/no-mistakes-tmp/no-mistakes-evidence/01KYJ416152RY1AEM1GEHQVFYR/drive-panel-demo.sh)
Evidence: Key transcript excerpt: analysts on different models, judge held back, then dispatched and completed; and the single-model refusal
$ bin/fm-model-panel.sh start --id q1 ... --question-file question.md
panel: q1 form=panel project=$HOME/subject
dispatched: q1-a harness=claude model=claude-opus-5 effort=xhigh
dispatched: q1-b harness=codex model=gpt-5.6-sol effort=xhigh
panel: next step is `... advance q1` once every analyst reports done

$ bin/fm-model-panel.sh advance q1
waiting: the judge is created only once every analyst has finished AND left a non-empty report; q1-b left a report but has not written its terminal status event yet

$ bin/fm-model-panel.sh advance q1
dispatched: q1-judge harness=grok model=grok-4-fast effort=high
judge brief does not mention claude-opus-5: ok
judge brief does not mention gpt-5.6-sol: ok

$ bin/fm-model-panel.sh advance q1
complete: $HOME/data/q1-judge/report.md

$ bin/fm-model-panel.sh start --id d1 ... # both analysts resolve to one model
error: this would not be a panel: both analysts resolve to the model claude-opus-5.
or re-run with --reduced for the single-analyst review, which is labelled as such everywhere it appears.
[exit 4]

Pipeline

Updates from git push no-mistakes

... (4 earlier update rounds omitted to keep the PR body within GitHub's 65536-char limit; full history is in the run log.)

⚠️ **Review** - 1 info

🔧 Fix: gate judge report completion and add recorded unfinished override
4 issues (2 warnings, 2 infos) still open:

  • ⚠️ bin/fm-model-panel.sh:808 - The wedged: block ignores the liveness half of the condition it was asked to detect. The captain's instruction was "When a member has a non-empty report but no terminal status event, AND IT IS NO LONGER LIVE, advance must stop printing the bland waiting: line and instead print an actionable state"; panel_gate_wait instead prints wedged: whenever any member is unsignalled, and member_unsignalled_advice uses [ -f &#34;$STATE/$id.meta&#34; ] (line 781) only to vary the wording. Failure mode: an analyst writes data/p-a/report.md and then spends another turn revising it before appending done:; firstmate runs advance p as the skill's step 3 tells it to; instead of a wait line it gets wedged: plus a five-line block whose closing lines offer advance p --accept-unfinished p-a. Taking that offer during the normal write window dispatches the judge against a report that genuinely is still being written, which is the exact race this whole gate exists to close. The contradiction is visible in the source: line 775 calls this "the one state that may never clear on its own" while line 782 tells the operator it "may simply still be working", and the skill repeats both. Two existing assertions had to be relaxed for this (tests/fm-model-panel.test.sh:217 and :363 dropped assert_contains &#34;waiting:&#34;), which is the normal path proving it fires there. Gate the wedged:/waiting: choice on the runtime record that line 781 already reads.
  • ⚠️ bin/fm-model-panel.sh:765 - The finished-empty state is the one that provably never clears, and it is the one that gets the bland waiting line. A member with a terminal done:/failed: event and no report has stopped writing by the gate's own definition, so no further report can arrive. Failure mode: analyst A appends failed: ran out of budget without ever creating data/p-a/report.md; advance p prints waiting: ... p-a has finished but its report ... is empty or absent and the hint "after the next analyst finishes" on every future invocation; --accept-unfinished p-a correctly refuses because a missing report is never waived (line 873), so there is no escape at all, and nothing in the output or the skill tells the operator that this panel can only be stood down. This is the mirror image of the wedged misclassification: a state that cannot clear is described as one to keep waiting on. Give it its own terminal outcome naming the stand-down as the only option, in the same shape as the stage=incomplete branch at line 885.
  • ℹ️ bin/fm-model-panel.sh:878 - --accept-unfinished is validated and written to panel.meta at lines 865-881, before the case &#34;$stage&#34; at line 883 decides whether this advance can do anything at all. Failure mode: advance p --accept-unfinished p-a on a panel whose stage=incomplete records accepted_unfinished=p-a and prints the permanent-record notice, then dies at line 885 with "panel never finished dispatching"; the record now carries an acceptance for a panel that never dispatched an analyst. The same happens harmlessly but pointlessly on stage=complete. Since the durable record exists precisely to carry the verdict's provenance, an entry that never affected any verdict is noise in the one place the captain accepted the override for. Move the acceptance block below the stage dispatch, so it runs only on the stages where the gate is actually evaluated.
  • ℹ️ bin/fm-model-panel.sh:899 - An accepted analyst report is disclosed to its downstream consumer through the judge brief's "Reports accepted without their author finishing" section, but an accepted JUDGE report has no downstream consumer except firstmate, and the completion output is byte-identical whether or not the verdict itself was accepted unfinished. Failure mode: advance p --accept-unfinished p-judge prints the recorded-permanently notice on that one invocation, then complete: &lt;path&gt; with the standard "read that report, relay its findings" instruction; any later advance p takes the stage=complete branch at line 887 and prints only the unqualified complete: pair, so a firstmate that resumes on a fresh context reads a normal completion. The panel skill does instruct relaying the caveat, and panel.meta carries accepted_unfinished, so this is a small gap rather than a hole: echoing the caveat on the complete: output would close it at the point firstmate actually acts.

🔧 Fix: gate wedge on member liveness and stand down reportless panels
2 warnings still open:

  • ⚠️ bin/fm-model-panel.sh:763 - The gate's durability premise is false: bin/fm-teardown.sh:1232 removes state/&lt;id&gt;.status and state/&lt;id&gt;.meta in the same rm -f, so the two signals member_state uses to tell "gone without finishing" from "finished then cleaned up" disappear together. The header at line 89 chose the status event over a live crew-state read "because panel members are ordinary scouts that may be torn down before the judge is dispatched", and this comment reasons that .meta's absence is "the durable proof that no further line will ever be appended" - but teardown deletes both, so line 791 classifies a correctly-finished member as unsignalled-gone. Failure mode: analyst A writes data/p-a/report.md, appends done: to state/p-a.status, and firstmate tears it down to free the pane before running advance p; status_has_finished_event finds no file and member_still_present finds no file, so advance prints the wedged: block asserting A "never wrote a terminal status event" and that "it was torn down or died and that line will never arrive". The only way forward is --accept-unfinished p-a, which permanently records accepted_unfinished and adds the "Reports accepted without their author finishing" section to the judge's brief, telling the judge to treat a complete report as possibly truncated and falsifying the verdict's provenance in the exact field the override exists to keep honest. The same happens to the judge at stage=judge, putting a permanent CAVEAT on a complete verdict. Snapshotting each member's ready state into panel.meta the first time advance observes both conditions would make the fact as durable as the header claims it already is.
  • ⚠️ bin/fm-model-panel.sh:991 - The judge stage runs the same panel_gate_wait, so a judge that ends terminal with no report returns 2 and panel_stand_down records stage=stood-down permanently - discarding two complete analyst investigations because the third member crashed. The captain's rationale for the stand-down is analyst-specific ("a verdict built on the one report that survived is not a panel"), but panel_stood_down_block prints it verbatim for the judge at line 843, where both analyst reports exist and the formation is intact. Failure mode: the judge appends failed: ran out of budget without writing data/p-judge/report.md; advance p prints "panel p cannot be completed", tells the operator "--accept-unfinished deliberately refuses to waive a report that does not exist: a verdict built on what survives is not a panel", and directs the salvage to "a NEW single-analyst review started deliberately with start --reduced" - which would re-run an analyst from scratch rather than re-judge the two finished reports sitting in data/, and there is no longer any path to dispatch a replacement judge because the stage is terminal. The new stand-down test covers only the analyst case (sd-b), so this path is unexercised. Either scope the stand-down to analysts and give a dead judge a re-dispatch path, or keep it and make the block say what a dead judge actually costs and what to do about it.

🔧 Fix: latch panel readiness and replace a verdictless judge
3 issues (2 warnings, 1 info) still open:

  • ⚠️ bin/fm-model-panel.sh:1109 - --rejudge does not check that the current judge actually ended. The header at line 68 defines it as dispatching a replacement "after the current judge ended without writing a verdict", and panel_rejudge_block is only reached from the finished-empty gate result, but the flag itself guards on just two conditions: stage = judge (line 1107) and an empty report_judge (line 1109). A judge that was dispatched moments ago satisfies both. Failure mode: advance p prints waiting: ... p-judge has written no terminal status event yet ... and no report because the judge is still reading the reports; the operator runs advance p --rejudge; judge_task is repointed to p-judge2, a second judge is dispatched over the same reports, and the original agent keeps running and burning quota with nothing in the panel record referring to it, so if it later writes data/p-judge/report.md that verdict is silently ignored. Requiring a terminal status event for the current judge before superseding it would match the documented precondition. Flagging rather than fixing because the right test is a design call: a crashed judge keeps its state/&lt;id&gt;.meta, so a liveness-absence test would recreate the no-escape dead end the captain ruled against, exactly as it would for --accept-unfinished.
  • ⚠️ bin/fm-model-panel.sh:1117 - The rejudge branch writes superseded_judges and judge_task to panel.meta before it scaffolds or dispatches anything, unlike the judge-dispatch path below it, where panel_meta_set &#34;$meta&#34; stage judge is deliberately the last statement after a successful dispatch (line 1176). Failure mode: advance p --rejudge records superseded_judges=p-judge and judge_task=p-judge2, then dispatch_scout fails and line 1175 dies; the record now claims a judge was superseded when it was not, and points judge_task at a task that was never dispatched, so every later advance p gates on p-judge2 and prints waiting: ... has written no terminal status event yet ... and no report forever. Recovery costs another --rejudge, which burns a third id and records a second false supersede. Both writes can move below the successful dispatch_scout, next to the existing stage write: the scaffold and dispatch use the local $id_judge, not the record, so nothing between them reads the fields.
  • ℹ️ bin/fm-model-panel.sh:816 - The latch fixes the case it was scoped to, but the window is not zero: member_observed_ready can only return true while state/&lt;id&gt;.status exists, so a member that finishes and is torn down before any advance observes it is never latched. Failure mode: analyst A appends done:, firstmate relays and tears A down without running advance first, and the next advance p sees no status file, no runtime record, and a non-empty report - classifying A as unsignalled-gone, printing the wedged: block, and letting --accept-unfinished p-a stamp accepted_unfinished on a complete report and tell the judge to treat it as truncated. The panel skill's operating sequence puts advance at step 3 and teardown at step 7, and the done: event wakes firstmate before any teardown, so this is a bounded tradeoff of the design the captain approved rather than something to change. Noting it only because the new header block reads as though the durability premise is now fully repaired, and the residual depends on an ordering that lives in the skill rather than in the script.

🔧 Fix: guard rejudge against a live judge and commit after dispatch
2 issues (1 warning, 1 info) still open:

  • ⚠️ bin/fm-model-panel.sh:867 - member_state's no-report branch never consults member_still_present, so a member that is definitively gone without ever writing a report is classified working - identical to one that started moments ago. The report-present branch directly above (lines 861-865) correctly splits unsignalled-live from unsignalled-gone; this one does not, leaving the fourth quadrant with no handler. Failure mode: analyst A is killed hard or torn down before writing anything and without appending a terminal line, and bin/fm-teardown.sh:1232 removes state/p-a.status and state/p-a.meta together so nothing distinguishes it from a live analyst; advance p prints waiting: ... p-a has written no terminal status event yet ... and no report with the hint "after the next analyst finishes" on every future invocation; --accept-unfinished p-a correctly refuses because there is no report to waive; no stand-down fires because that requires the terminal event; and start --id p refuses because data/p exists, so the only recovery is deleting the panel directory by hand. This is the same root pattern the readiness latch and the new --rejudge precondition each corrected one layer along - the classification keys on an artifact teardown deletes - and the captain's round-3 reasoning for stood down: (the premise has failed, nothing further can arrive) applies identically to a gone member with no report. For a torn-down JUDGE in this quadrant the remedy already exists and succeeds (--rejudge is allowed once the runtime record is gone), but advance never offers it.
  • ℹ️ .agents/skills/panel/SKILL.md:92 - The skill's --rejudge section documents only the verdict lost: path, where the judge ended with a terminal status event and no report. It has no counterpart to the sentence the --accept-unfinished section carries for exactly the analogous case: "It also stays available for a member that still looks present but you know is dead, such as one killed by a harness crash that left its runtime record behind." Failure mode: the judge crashes holding state/p-judge.meta without writing a verdict or a terminal line; advance p prints the plain waiting: line, which is correct because that state is indistinguishable from a working judge; firstmate consults the skill, finds nothing covering it, and waits indefinitely. The recovery - tear the judge down, then --rejudge, which the new precondition deliberately permits once the runtime record is gone - appears only in the refusal message the operator sees by guessing to try the flag. One sentence mirroring the --accept-unfinished paragraph would close it.

🔧 Fix: classify gone reportless members and state the root cause
1 info still open:

  • ℹ️ bin/fm-model-panel.sh:891 - The finished-empty and gone-empty arms of member_wait_reason can never be printed. Both states push the member onto empty in panel_gate_wait (lines 978-979), and the [ &#34;${#empty[@]}&#34; -gt 0 ] check at line 983 returns 2 before either the wedged: or waiting: line that consumes reasons is emitted, so the strings are built and discarded on every such call. The wording an operator actually sees for those two states comes from panel_stood_down_block (line 931) and panel_rejudge_block (line 947), each of which hand-writes its own version of the same sentence. Failure mode: a maintainer edits the gone-empty wording here to change what the stand-down says, sees no effect, and either duplicates the change into the block or leaves two divergent descriptions of one state in the file - the finished-empty arm has already been dead since the stand-down landed in round 4, and this round added a second one beside it. Either drop the two arms, or have panel_stood_down_block and panel_rejudge_block take the reason strings the gate already computed so one function owns each state's wording.
✅ **Test** - passed

✅ No issues found.

  • bin/fm-test-run.sh tests/fm-model-panel.test.sh tests/fm-gate-refuse.test.sh - 23 panel behavior cases plus the gate-refusal suite including the new fm-model-panel entry point, all passing
  • bin/fm-test-run.sh --check-coverage and bin/fm-test-run.sh --list --family pure-contract-unit - confirms tests/fm-model-panel.test.sh is registered in the pure-contract-unit family and the coverage guard is satisfied
  • Manual end-to-end drive of the real bin/fm-model-panel.sh in a sandbox home (only bin/fm-spawn.sh stubbed to record argv, FM_DISPATCH_QUOTA_AXI pointed at an absent command): start --dry-run, start --question-file, four successive advance calls across the analyst and judge gates through complete:, and status
  • Manual verification that the two analyst briefs differ only in their A/B label and their own task paths (diff of the generated briefs) and that the judge brief contains none of the configured model or harness names
  • Manual degradation drive: start with two harnesses resolving to one model identity (claude/claude-opus-5 vs pi/anthropic/claude-opus-5:1m) refusing with exit 4, then start --reduced plus advance showing the single-analyst-review labelling in the record, the analyst brief, and the judge brief
  • Manual configuration drive: no-config refusal naming both config files, crew-dispatch.json default-profile fallback, absence of model-panel.json from FM_INHERITABLE_CONFIG, and the no-mistakes gate refusal (exit 3) with the test bypass removed
⚠️ **Document** - 1 info
  • ℹ️ bin/fm-config-inherit-lib.sh:29 - bin/fm-config-inherit-lib.sh's header enumerates the deliberately-excluded inheritable-config items with reasons (config/secondmate-harness, config/fork-sync-upstream, config/role) but not config/model-panel.json. I deliberately did not add it: config/backend is likewise excluded and only documented in docs/configuration.md, and that doc's per-config sections own the inheritance fact, so extending the header would create a second prose copy. If a future change wants that header to be an exhaustive exclusion list, that is a small consolidation to do for backend and model-panel together, in the header or by reducing it to a pointer.
✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Two analysts answer one question independently on different models without
seeing each other's work, then a third model judges both reports and
re-verifies their load-bearing claims itself. The value is independence plus
adversarial verification, not three opinions, so the formation is now a script
plus a skill instead of something firstmate hand-assembles each time.

bin/fm-model-panel.sh dispatches every member as an ordinary scout through the
existing brief, spawn, and report lifecycle, and invents no second one. `start`
resolves the lineup and dispatches the analysts concurrently; `advance` creates
the judge only once every analyst report exists, and each non-terminal outcome
names the next command so one entry point carries a panel from question to
answer.

Roles are pinned in code, models are not. config/model-panel.json maps each role
to a dispatch profile resolved through bin/fm-dispatch-select.sh, so panel
profiles get the same validation and quota-aware array selection as crew
dispatch profiles, and an unconfigured home falls back to the crew-dispatch
default profile set. A model name in tracked code would rot silently, and config
is also what lets each home run a panel on the models it can actually reach,
which is why the file is deliberately not inherited by secondmate homes.

Degradation is explicit. When both analysts would resolve to one model identity
- including the same model reached through two harnesses - start refuses and
names both the configuration fix and the opt-in reduced form, because a panel
that quietly is not one is worse than no panel. The reduced form is recorded and
labelled as a single-analyst review in the briefs, the panel record, and the
judge's own report.

The briefs carry the discipline that made the hand-runs pay: verify status prose
against live state rather than trusting it, check your vantage point before
concluding anything about the environment, label your highest-conviction calls
so the judge can score them, and inventory unresolved captain decisions. The
judge is told to re-verify rather than referee, to hunt for the mistake both
sides made for the same reason, and judges the reports blind of which model
wrote which. The scout scaffold's decision-hold completion gate carries through
to every member.
@Freudator86
Freudator86 merged commit 6bc798a into main Jul 27, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant