Context
Follow-up from #405 / PR #406. Cut from that PR's review round with a reason, recorded here.
0.9.1 ships two presence probes for the renderer-era dev primitive (BMAD-METHOD#2601):
skills.dev-renderer (is _bmad/scripts/render_skill.py there?) and
skills.dev-renderer-config (is _bmad/config.toml there?). Neither answers the question an
operator actually has — will the renderer render? Everything past file presence is unprobed:
a RenderError: missing config value ..., an ambiguous config value ... found at: ..., an
unreadable customize.toml, a ModuleNotFoundError from a half-installed _bmad/scripts/, or
uv failing to resolve. Each of those is the same result-less Stop: the stub writes HALT: ...
to stdout, exits 1, and the session ends with no spec on disk.
Proposal
bmad-loop validate --render-probe — opt-in flag, off by default — executes the renderer for
real, once per resolved stub, and reports its actual verdict.
Constraints (the reason this is a flag and not a default check)
-
It must use the stub's own argv. The real invocation lives in the stub's SKILL.md:
uv run --no-cache "{project-root}/_bmad/scripts/render_skill.py" --project-root "{project-root}" --skill "{skill-root}"
{project-root} and {skill-root} are substituted by the agent, not by us. A probe that
guesses the argv validates a different command than the session runs — green would be a lie
exactly where it matters most. Parse the command line out of the resolved SKILL.md and
substitute the same two tokens; if the line cannot be parsed, report that rather than falling
back to a hand-built argv.
-
It makes validate a writer. render() publishes into
_bmad/render/<skill>/<slug>-<root-hash>/<generation-hash>/ (render_skill.py), so a probe
mutates the project. That is the whole reason it is opt-in. Note it in --help, and note that
0.9.1 already gitignores _bmad/render/ (init) and git-excludes it inside worktrees.
-
Contract to key on: stdout read and follow <path> + rc 0 on success; HALT: <error> +
rc 1 on failure. A bare traceback with neither is its own finding — it means the failure
escaped the renderer's try/except (the ModuleNotFoundError shape).
-
Severity: FAIL when the probe was requested and the renderer HALTs; the probe must not run
under plain validate, so the existing severity ladder is untouched.
-
uv on PATH is in scope for the probe only. The validate host's PATH still does not answer
the session's PATH question (sessions run in fresh panes), so a probe failure on
"uv not found" should say which PATH it searched.
Context
Follow-up from #405 / PR #406. Cut from that PR's review round with a reason, recorded here.
0.9.1 ships two presence probes for the renderer-era dev primitive (BMAD-METHOD#2601):
skills.dev-renderer(is_bmad/scripts/render_skill.pythere?) andskills.dev-renderer-config(is_bmad/config.tomlthere?). Neither answers the question anoperator actually has — will the renderer render? Everything past file presence is unprobed:
a
RenderError: missing config value ..., anambiguous config value ... found at: ..., anunreadable
customize.toml, aModuleNotFoundErrorfrom a half-installed_bmad/scripts/, oruv failing to resolve. Each of those is the same result-less Stop: the stub writes
HALT: ...to stdout, exits 1, and the session ends with no spec on disk.
Proposal
bmad-loop validate --render-probe— opt-in flag, off by default — executes the renderer forreal, once per resolved stub, and reports its actual verdict.
Constraints (the reason this is a flag and not a default check)
It must use the stub's own argv. The real invocation lives in the stub's
SKILL.md:{project-root}and{skill-root}are substituted by the agent, not by us. A probe thatguesses the argv validates a different command than the session runs — green would be a lie
exactly where it matters most. Parse the command line out of the resolved
SKILL.mdandsubstitute the same two tokens; if the line cannot be parsed, report that rather than falling
back to a hand-built argv.
It makes
validatea writer.render()publishes into_bmad/render/<skill>/<slug>-<root-hash>/<generation-hash>/(render_skill.py), so a probemutates the project. That is the whole reason it is opt-in. Note it in
--help, and note that0.9.1 already gitignores
_bmad/render/(init) and git-excludes it inside worktrees.Contract to key on: stdout
read and follow <path>+ rc 0 on success;HALT: <error>+rc 1 on failure. A bare traceback with neither is its own finding — it means the failure
escaped the renderer's
try/except(theModuleNotFoundErrorshape).Severity: FAIL when the probe was requested and the renderer HALTs; the probe must not run
under plain
validate, so the existing severity ladder is untouched.uvon PATH is in scope for the probe only. The validate host's PATH still does not answerthe session's PATH question (sessions run in fresh panes), so a probe failure on
"uv not found" should say which PATH it searched.