Skip to content
This repository was archived by the owner on Feb 24, 2026. It is now read-only.

Commit 8521bfa

Browse files
seanphanseanphan
committed
docs: add minion runbook, ops coverage reports, and planning docs
Add closed-loop SOP, definition of done, n8n quickwin results, SDK delegation plan, and ops coverage reports.
1 parent a55bb97 commit 8521bfa

18 files changed

+2117
-0
lines changed

‎.python-version‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.12.1

‎docs/minion/closed_loop_sop.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Minion Closed-Loop SOP
2+
3+
This SOP defines an autonomous execution loop for minion workers and QA.
4+
All autonomous runs must follow this sequence until DoD is met.
5+
6+
## Phase Loop
7+
8+
Plan -> Build -> Test -> Semantic Verify -> QA -> Loop/Retry -> Ship
9+
10+
1. Plan
11+
- Read task brief, source references, and current `docs/minion/definition_of_done.md`.
12+
- Record assumptions, out-of-scope items, and risks.
13+
2. Build
14+
- Apply focused changes only for the assigned task.
15+
- Record files touched and command(s) planned.
16+
3. Test
17+
- Run task-relevant checks (lint/tests/CLI checks as applicable).
18+
- Capture command + exit code + key output.
19+
4. Semantic Verify
20+
- Validate behavior against intended semantics, not only transport success.
21+
- Confirm capability coverage, tool-backed execution, and expected outputs.
22+
5. QA
23+
- Validate outputs against DoD and rejection rules.
24+
- Verify release-readiness gate command is runnable and passing.
25+
6. Loop/Retry
26+
- If any phase fails, classify and route via failure matrix.
27+
- Apply remediation and re-run Plan→...→QA.
28+
7. Ship
29+
- Stop only when all stop conditions are satisfied.
30+
31+
## Stop Conditions
32+
33+
- Stop and hand over as **DONE** when:
34+
- Workflow and/or agent checks satisfy `docs/minion/definition_of_done.md`.
35+
- Semantic Verify is pass/fail documented as `pass`.
36+
- QA confirms release readiness and no hard blockers remain.
37+
- Stop and escalate as **BLOCKED** when:
38+
- Max retry policy is exhausted.
39+
- Required artifact is missing or unrecoverable in this loop.
40+
- Security/compliance or dependency constraints cannot be resolved.
41+
42+
## Max Retry Policy
43+
44+
- Retry budget per task: 3 total full loop attempts.
45+
- Per attempt, rerun only failed phases after fixes.
46+
- After 3 failed full attempts, stop as BLOCKED and report blockers.
47+
48+
## Artifact Requirements per Iteration
49+
50+
Each loop attempt must produce:
51+
52+
- `Plan`: task hypothesis and changed file list.
53+
- `Build`: code/docs diff and command plan.
54+
- `Test`: command list with pass/fail status.
55+
- `Semantic Verify`: explicit pass/fail against intended behavior.
56+
- `QA`: DoD check against `docs/minion/definition_of_done.md` with blockers.
57+
- `Release`: output of `bash scripts/release_readiness.sh` (or explicit failure reason).
58+
59+
## Failure Classification and Remediation Routing
60+
61+
- `Transport-only pass`
62+
- Root cause: command exit status passed but intent/semantic checks failed.
63+
- Route: rerun with semantic checks and user-behavior evidence.
64+
- `Semantic fail`
65+
- Route: fix behavior mismatch and re-run Test + Semantic Verify.
66+
- `Test failure`
67+
- Route: fix implementation defect, missing deps, or fixture issues; rerun failing tests.
68+
- `Tooling/infra failure`
69+
- Route: capture error context, retry once, then escalate if persistent.
70+
- `Blocked/dependency`
71+
- Route: mark BLOCKED immediately and escalate unresolved items.
72+
73+
## Release Gate Requirements
74+
75+
Before Ship, all runs must satisfy:
76+
77+
1. `bash scripts/release_readiness.sh` completes successfully.
78+
2. DoD checks in this repo are met for affected scope.
79+
3. Semantic checks are explicitly marked pass (no placeholders).
80+
4. Artifacts for the iteration are present and internally consistent.
81+
5. Transport-only and silent semantic regressions are rejected.

‎docs/minion/definition_of_done.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Minion Definition of Done
2+
3+
This file defines what workers and QA must enforce before calling a workflow/agent task complete.
4+
5+
## Workflow Done
6+
7+
All required:
8+
9+
1. Shape checks pass
10+
- `agenticflow workflow validate --body @workflow.json`
11+
2. Lifecycle checks pass
12+
- Create/update returns success
13+
- Get/read confirms persisted entity
14+
3. Runtime checks pass
15+
- Run returns `workflow_run_id`
16+
- Run status reaches terminal `success`
17+
4. Semantic checks pass
18+
- Output satisfies source intent, not just generic model response
19+
- Tool-backed intent requires tool-backed behavior
20+
5. Evidence provided
21+
- Payload(s), run id, final status payload, and short pass/fail table
22+
23+
## Agent Done
24+
25+
All required:
26+
27+
1. `agent create` succeeds
28+
2. `agent get` returns created agent
29+
3. `agent update` succeeds
30+
4. `agent stream` succeeds with at least one real prompt
31+
5. If tool use is expected, at least one test proves tool-backed behavior
32+
6. Evidence provided: payloads, ids, transcript snippets, pass/fail table
33+
34+
## Rejection Rules
35+
36+
Reject as not done if any apply:
37+
38+
1. Only dry-run evidence is provided.
39+
2. Only transport success is shown (no semantic verification).
40+
3. Required capabilities from source template are silently dropped.
41+
4. Errors are reported without actionable remediation.

‎docs/minion/tasks/qa-release.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
You are QA release minion for agenticflow-cli.
2+
3+
Mission:
4+
Run the CLI test suites and release gates, then return a strict PASS/FAIL verdict for release-readiness.
5+
6+
Required commands (in this order):
7+
1) `PATH=/Users/sean/.nvm/versions/node/v22.18.0/bin:$PATH bash scripts/release_readiness.sh`
8+
2) `PYTHONPATH=. .venv/bin/python -m pytest -q tests/unit`
9+
3) CLI smoke checks:
10+
- `PYTHONPATH=. .venv/bin/python scripts/agenticflow_cli.py --help`
11+
- `PYTHONPATH=. .venv/bin/python scripts/agenticflow_cli.py code search --help`
12+
- `PYTHONPATH=. .venv/bin/python scripts/agenticflow_cli.py code execute --help`
13+
- `node ./bin/agenticflow.js --help`
14+
15+
Output requirements:
16+
1) Provide PASS/FAIL.
17+
2) Provide exact commands run.
18+
3) If FAIL, provide top blockers with file/line if applicable.
19+
4) If PASS, confirm the package is ready for version bump + publish workflow run.

‎docs/minion/tasks/qa.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
You are QA minion for agenticflow-cli.
2+
3+
Mission:
4+
Validate remediation outputs against docs/solid_plan.md and docs/minion/definition_of_done.md.
5+
6+
Required checks:
7+
1) Run `PATH=/Users/sean/.nvm/versions/node/v22.18.0/bin:$PATH bash scripts/release_readiness.sh`.
8+
2) Verify closed-loop harness latest artifact no longer fails on empty node validation and reaches create/run/poll path.
9+
3) Verify latest runtime report contains runtime + semantic verdict with evidence.
10+
4) Verify ops coverage uses declared supported baseline and avoids placeholder UUID parsing failures for supported executed ops.
11+
5) Verify coverage report exists and classifies each attempted operation.
12+
13+
Acceptance policy:
14+
- Reject transport-only success.
15+
- Reject missing semantic evidence.
16+
- Produce final verdict: PASS or FAIL with blockers and exact files/commands.

‎docs/minion/tasks/worker-1.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
You are worker-1 (`runtime-loop-fix`) for agenticflow-cli.
2+
3+
Mission:
4+
Fix closed-loop harness so real template 6270 produces a non-empty workflow payload and reaches create/run/poll stages with real key.
5+
6+
Scope:
7+
1) Inspect live response shape from:
8+
- `GET /v1/workflow_templates/6270`
9+
2) Fix `scripts/runtime_loop_harness.py` template extraction logic to handle current live schema robustly.
10+
3) Add defensive fallback when extracted nodes are empty:
11+
- fail loud with explicit reason, or
12+
- auto-select a safe minimal runnable node only if deterministic.
13+
4) Preserve structured artifact output format.
14+
5) Add/adjust tests if testable without live network.
15+
16+
Required validation:
17+
1) Run:
18+
- `set -a; source /Users/sean/WIP/Antigravity-Workspace/WorkflowChef-Web/.env; set +a`
19+
- `PYTHONPATH=. .venv/bin/python scripts/runtime_loop_harness.py --template-id 6270`
20+
2) Provide artifact paths and verdict fields.
21+
22+
Acceptance:
23+
1) Harness no longer fails at validate due empty `nodes`.
24+
2) At least one attempt reaches create/run/poll stages.
25+
3) Report contains runtime + semantic verdict and clear evidence.
26+
4) Provide changed files and commands run.

‎docs/minion/tasks/worker-2.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
You are worker-2 (`ops-coverage-fix`) for agenticflow-cli.
2+
3+
Mission:
4+
Fix ops coverage harness so declared supported operations are evaluated realistically with real key, not mostly placeholder-validation failures.
5+
6+
Scope:
7+
1) Review `scripts/ops_coverage_harness.py` declared operation set and execution policy.
8+
2) Implement fixture/bootstrap strategy for IDs:
9+
- resolve real UUID workspace/project if possible,
10+
- create/find temporary workflow/agent/thread/run when needed,
11+
- avoid fake `*_demo` IDs for operations that require UUID path params.
12+
3) Separate operation support levels clearly:
13+
- executed
14+
- blocked-by-policy
15+
- unsupported
16+
4) Ensure failures represent real auth/infra/semantic problems, not avoidable placeholder errors.
17+
5) Update `docs/ops_coverage_report.{json,md}` from a real-key run.
18+
19+
Required validation:
20+
1) Run:
21+
- `set -a; source /Users/sean/WIP/Antigravity-Workspace/WorkflowChef-Web/.env; set +a`
22+
- `PYTHONPATH=. .venv/bin/python scripts/ops_coverage_harness.py --env-file /Users/sean/WIP/Antigravity-Workspace/WorkflowChef-Web/.env --report-json docs/ops_coverage_report.json --report-md docs/ops_coverage_report.md`
23+
2) Print summary totals and classification counts.
24+
25+
Acceptance:
26+
1) No avoidable `uuid_parsing` failures remain for supported executed ops.
27+
2) Report classifications are actionable and exact by operation id.
28+
3) Provide changed files and commands run.

‎docs/minion/tasks/worker-2b.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
You are worker-2b (`ops-harness-single-owner`) for agenticflow-cli.
2+
3+
Mission:
4+
Finish `scripts/ops_coverage_harness.py` to a correct, executable state and produce final real-key coverage reports.
5+
6+
Rules:
7+
1) You are the ONLY worker editing `scripts/ops_coverage_harness.py` in this run.
8+
2) Keep compatibility with existing report schema where possible.
9+
3) Ensure script compiles and runs.
10+
4) Ensure support-scope constants are internally consistent (no undefined names).
11+
5) Execute harness with real key env:
12+
- set -a; source /Users/sean/WIP/Antigravity-Workspace/WorkflowChef-Web/.env; set +a
13+
- PYTHONPATH=. .venv/bin/python scripts/ops_coverage_harness.py --env-file /Users/sean/WIP/Antigravity-Workspace/WorkflowChef-Web/.env --report-json docs/ops_coverage_report.json --report-md docs/ops_coverage_report.md
14+
6) Print summary totals and classification counts.
15+
16+
Acceptance:
17+
- script runs without syntax/runtime crash
18+
- docs/ops_coverage_report.json + .md regenerated
19+
- final message includes changed files and commands

‎docs/minion/tasks/worker-3.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
You are worker-3 (`qa-remediation`) for agenticflow-cli.
2+
3+
Mission:
4+
Run release gate + harnesses after worker fixes and produce a strict PASS/FAIL verdict with blockers.
5+
6+
Scope:
7+
1) Pull latest local changes in your working tree.
8+
2) Run release gate:
9+
- `PATH=/Users/sean/.nvm/versions/node/v22.18.0/bin:$PATH bash scripts/release_readiness.sh`
10+
3) Run closed-loop harness with real key:
11+
- `set -a; source /Users/sean/WIP/Antigravity-Workspace/WorkflowChef-Web/.env; set +a`
12+
- `PYTHONPATH=. .venv/bin/python scripts/runtime_loop_harness.py --template-id 6270`
13+
4) Run ops coverage harness with real key:
14+
- `PYTHONPATH=. .venv/bin/python scripts/ops_coverage_harness.py --env-file /Users/sean/WIP/Antigravity-Workspace/WorkflowChef-Web/.env --report-json docs/ops_coverage_report.json --report-md docs/ops_coverage_report.md`
15+
5) Write final verdict summary with exact file paths.
16+
17+
Acceptance:
18+
1) Report includes:
19+
- release gate result
20+
- runtime/semantic verdict from latest runtime artifact
21+
- ops totals + classification counts
22+
2) Final output is PASS only if all required gates in `docs/solid_plan.md` are satisfied.

‎docs/minion/tasks/worker-4.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
You are worker-4 (`support-matrix`) for agenticflow-cli.
2+
3+
Mission:
4+
Define and enforce a precise "supported operations baseline" so coverage reflects intentional support, not all raw public spec endpoints.
5+
6+
Scope:
7+
1) Audit:
8+
- `src/agenticflow_cli/operation_ids.py`
9+
- `src/agenticflow_cli/public_ops_manifest.json`
10+
- `scripts/ops_coverage_harness.py`
11+
2) Implement a support matrix model (docs + code) that clearly marks each op as:
12+
- supported-executed
13+
- supported-blocked-policy
14+
- out-of-scope
15+
3) Ensure harness and release docs use this same baseline.
16+
4) Update documentation with explicit rationale for each class.
17+
18+
Constraints:
19+
- Do not over-claim support for endpoints not wrapped or not safely executable by CLI.
20+
- Keep backward compatibility for existing wrapper commands.
21+
22+
Acceptance:
23+
1) There is a single source of truth for supported coverage scope.
24+
2) Coverage report + docs align with that scope.
25+
3) Provide changed files and commands run.

0 commit comments

Comments
 (0)