feat: broaden E2E visual review trigger and add review-page build skill - #3
Merged
Conversation
added 5 commits
July 29, 2026 02:26
Whenever a task actually runs a project's E2E test suite, it now captures screenshots (and video where the framework supports it) and leaves that evidence at data/<task-id>/e2e-review/ in the firstmate home, a third named exception to worktree isolation alongside report.md and the status file. Building the actual review page from that evidence stays firstmate's own job. Mirrors the project-docs/version-management pattern: a new agent-only skill (e2e-visual-review) owns the full protocol, with pointers from the ship-brief scaffold and AGENTS.md section 7.
…inventory for E2E visual review
…skill Broaden the trigger from literal E2E-suite-only to any test run that exercises the running application (E2E, UI/browser-driven, visual regression) - plain unit/logic tests still just report pass/fail. Updates e2e-visual-review's scope text, both ship/scout brief pointers, and the AGENTS.md layout/pointer lines to match; also fixes the ship brief's Rule 2, which still said "modify nothing outside it" despite its own E2E Visual Review worktree exception (the scout brief already had this fixed by the prior no-mistakes document step). Closes the other half of the convention: firstmate never had a documented procedure for actually building the captain's review page from evidence crewmates left at data/<task-id>/e2e-review/. Adds .agents/skills/app-test-review/SKILL.md (agent-only, firstmate-side) covering when/where/contents/mechanism for building an interactive Lavish page at .lavish/<project>/<type>-<task-id>/review.html, wired into AGENTS.md section 7's PR-ready handling and section 13's trigger index, plus a new .lavish/ layout entry in section 2. Keeps the e2e-visual-review name rather than renaming: the scope text is now unambiguous on its own, and a rename would touch more surface (paths, tests, cross-references) for no behavioral gain.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Broaden the fleet-wide E2E visual review convention (captain-authorized amendment on top of the already-checks-passed PR #3) in two ways, applied on the same branch/PR as an amendment, not a new task: (1) Broaden the trigger from literal 'runs the project's E2E test suite' to any test run that exercises the running application - E2E suites, UI/browser-driven tests, and visual regression checks - explicitly excluding plain unit/logic tests, which have nothing meaningful to screenshot and keep reporting pass/fail as before. Updated: .agents/skills/e2e-visual-review/SKILL.md's Scope section and frontmatter description; both '# E2E Visual Review' heredoc sections in bin/fm-brief.sh (ship and scout); AGENTS.md's data//e2e-review/ layout description and its plain-language pointer sentence in section 7. Decision: kept the skill's name e2e-visual-review rather than renaming it (e.g. to something like app-visual-review) - the scope text itself is now unambiguous about covering more than literal E2E suites, and a rename would touch more surface (directory path, brief pointers, doc-audience inventory, tests) for no behavioral gain; this was a deliberate judgment call the captain explicitly left to me ('your call - a rename is not required if the scope text alone makes it clear, but flag your reasoning either way'). (2) Closed a real gap the captain hit in practice: no documented procedure existed for firstmate to actually build the captain-facing review page from evidence crewmates leave at data//e2e-review/ - the crewmate skill explicitly stops at 'producing evidence; building the page is firstmate's job' but nothing said how or when. Added a new agent-only skill .agents/skills/app-test-review/SKILL.md (same shape/frontmatter as other section-13 skills: internal:true, user-invocable:false) specifying: firstmate builds the review page whenever a task with data//e2e-review/ evidence reaches a captain-facing checkpoint (PR ready or done) - the default artifact, not optional; the page lives at .lavish//-/review.html inside firstmate's own home (captain-private, gitignored, durable across worktree teardown, unlike the source evidence which does not survive /stow-style cleanup the same way), where is feature-review or bug-detection inferred from the change; screenshots/video must be embedded directly (data: URIs or otherwise self-contained per Lavish's own constraints), never merely linked, using the crewmate's manifest.md to caption/order them; built via lavish-axi per its own conventions (playbook guidance, help output) so the captain gets the normal interactive Lavish review surface rather than a static dump. Wired this new skill into AGENTS.md: a new .lavish//... entry in section 2's layout listing, an inline pointer in section 7's 'PR ready, landing, and teardown' subsection (the exact captain-facing checkpoint where it must fire), and a section-13 trigger-index entry mirroring decision-hold-lifecycle's dual placement (both a section-13 entry and an inline pointer at its precise operational trigger). Also fixed, while already touching this exact wording: the ship brief's Rule 2 still said 'Stay inside this worktree; modify nothing outside it', directly contradicting its own later E2E Visual Review section which already declares data//e2e-review/ a permitted worktree-isolation exception; the scout brief had already been corrected for this same contradiction by the prior no-mistakes document-step fix on this branch, so I brought ship's Rule 2 wording in line with that same established pattern. Added regression-locking test assertions in tests/fm-brief.test.sh for the broadened scope wording and the corrected ship Rule 2 text in both existing ship/scout E2E-pointer tests. Updated docs/documentation-audiences.json's surface inventory for the new app-test-review skill (required by this repo's own fm-doc-audience-check.sh exact-coverage gate). Scope stayed limited to firstmate's own shared tracked material - no files under projects/ were touched, consistent with the original task.
What Changed
.agents/skills/e2e-visual-review/SKILL.md's Scope section and frontmatter description, both ship/scout# E2E Visual Reviewheredoc sections inbin/fm-brief.sh, andAGENTS.md'sdata/<id>/e2e-review/layout description and section 7 pointer sentence..agents/skills/app-test-review/SKILL.md(internal:true, user-invocable:false) documenting the previously-undefined procedure for firstmate to build the captain-facing review page from crewmate evidence atdata/<task-id>/e2e-review/, landing it at.lavish/<project>/<type>-<task-id>/review.htmlwith self-contained embedded media, built via lavish-axi; wired intoAGENTS.mdwith a section 2 layout entry, an inline section 7 pointer at the PR-ready/landing/teardown checkpoint, and a section 13 trigger-index entry.data/<task-id>/e2e-review/worktree-isolation exception, aligning it with the prior fix already applied to the scout brief.tests/fm-brief.test.shfor the broadened scope wording and corrected ship Rule 2 text, and registered the newapp-test-reviewskill indocs/documentation-audiences.jsonto satisfy the exact-coverage doc-audience gate.Risk Assessment
✅ Low: Doc/skill/brief-text-only change with an accompanying gate script (fm-doc-audience-check.sh) that already passes; every wording change (broadened E2E scope, ship Rule 2 fix, new app-test-review skill, section 2/7/13 wiring, documentation-audiences.json entries, new regression tests) matches the stated intent exactly, no code paths or runtime behavior beyond static text are affected, and no files under projects/ were touched.
Testing
Ran the targeted fm-brief and documentation-audiences test suites (all pass, including the two new regression tests locking in the broadened scope wording and fixed ship Rule 2 text), ran the repo's exact-coverage doc-audience gate script directly (passes with the two new skills registered), and manually generated real ship/scout briefs to visually confirm the actual crewmate-facing text now carries the broadened app-facing-test scope and the corrected, non-contradictory Rule 2 wording in both modes. This is a pure documentation/agent-instruction change with no browser-rendered UI surface, so the generated-brief text transcript (the actual artifact an end-user crewmate reads) serves as the closest available product-level evidence; it's saved alongside command output for reviewer reference. No test failures, no issues, and the working tree is clean.
Evidence: Actual generated ship/scout brief text (Rules + E2E Visual Review sections)
Evidence: tests/fm-brief.test.sh full run (17/17 pass)
Evidence: fm-doc-audience-check.sh exact-coverage gate
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
bash tests/fm-brief.test.sh— all 17 tests pass, including newtest_ship_e2e_visual_review_pointerandtest_scout_e2e_visual_review_pointerregression testsbash tests/fm-documentation-audiences.test.sh— all 5 tests passbash bin/fm-doc-audience-check.sh— exact-coverage gate passes (ok surfaces=59 local_links=147), confirming the new app-test-review and e2e-visual-review skills are correctly registered in docs/documentation-audiences.jsonManual verification: ranbin/fm-brief.shin both ship and scout mode against a scratch FM_HOME, and inspected the actual generated brief.md textCross-checked AGENTS.md for the claimed dual placement (section 2 layout entries at lines 85/116, section 7 inline pointers at lines 283/318, section-13 trigger entry at line 487) — all present as described in the intentConfirmedgit diff 05fba80..5c8a856 --stattouches only .agents/skills/app-test-review/SKILL.md, .agents/skills/e2e-visual-review/SKILL.md, AGENTS.md, bin/fm-brief.sh, docs/documentation-audiences.json, tests/fm-brief.test.sh — no files under projects/ touched✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.