Split Mind prompt-file lifecycle into draft/ active/ complete/

[Jammy2211]

Overview

PyAutoMind/issued/ is a flat, append-only pile of 444 prompt files — every prompt ever start_dev'd, done or not — so it is useless for tracking state. The root cause is two parallel lifecycles that have drifted apart: prompt files flow <work-type>/ → issued/ (and then never move), while ledger entries flow active.md → complete.md. This task makes the file lifecycle track the three ledger states via a clean top-level split — draft/ → active/ → complete/YYYY/MM/ — and splits the 6000-line complete.md into per-task rich records (the substrate for a later token-light archive wiki, filed separately).

Phase 2 (the complete/ wiki index) is the already-filed follow-up feature/pyautomind/complete_archive_wiki.md and is out of scope here — do not issue it until this nears shipping.

Plan

Ships as two sequential PRs under this issue (the Feature Agent flagged the task too-large for one PR; its generic core_api/workspace/docs phases don't apply to a Mind-internal restructure):

PR-A — Rewire (no historical data moved):

• Establish draft/ / active/ / complete/YYYY/MM/ conventions (zero-padded months = numerical order). draft/ wraps the existing <work-type>/<target>/ taxonomy.
• New scripts/lifecycle.py: move (bucket a completed file by its ledger completed: date), split-complete (complete.md → per-file records), --check (drift detection).
• Update the 6 Brain skills and 3 Mind scripts to the new paths (see Implementation Steps).
• Smoke a throwaway create_issue → ship round-trip. New work flows correctly; the 444 files still sit in issued/.

PR-B — Migrate (mechanical, big diff):

• Run lifecycle.py to move the 444 issued/ files into active/ vs complete/YYYY/MM/ (cross-referenced against active.md/complete.md; orphans → complete/unknown/), and split complete.md into per-file rich records — via a review manifest first, then git mv to preserve history.
• Regenerate + republish the spawn template (spawn --check / spawn_drift CI).

Detailed implementation plan

Affected Repositories

• PyAutoMind (primary) — dir layout, scripts/lifecycle.py, spawn.py, status.sh, prompt_sync.sh
• PyAutoBrain — the 6 dev-workflow skills wired to issued/ / complete.md

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoMind	main	clean
./PyAutoBrain	main	clean

Suggested branch: feature/lifecycle-state-split

Note: two stray codex worktrees sit on PyAutoMind branches (codex-organ-skill-wrappers, codex-skill-bookkeeping) — not conflicting (worktree_check_conflict exit 0), but land this before they grow to avoid merge friction.

Implementation Steps (PR-A)

1. Conventions + docs — define draft/ active/ complete/YYYY/MM/ in PyAutoMind/README.md "Prompt taxonomy"; add complete/AGENTS.md stub for the archive schema.
2. scripts/lifecycle.py — move <file> (read ledger completed: date → complete/<year>/<month>/), split-complete (parse complete.md H2 sections → complete/YYYY/MM/<slug>.md rich records + append ## Original prompt), --check (active-file whose ledger entry is already in complete.md; complete-file with no ledger entry; draft with an open issue; mis-bucketed date). Mirror repos_sync.py --check exit-code convention.
3. Brain skills (PyAutoBrain/skills/):
   • create_issue/SKILL.md — mv <path> issued/ → active/; "timestamp-suffix if exists in issued/" → active/.
   • ship_library/{ship_library,reference}.md + ship_workspace/{ship_workspace,reference}.md — on merge, also move the file active/<name>.md → complete/<year>/<month>/<slug>.md and write the rich record, in the same beat as the active.md → complete.md ledger move.
   • start_dev/reference.md — issued/<basename> lookup → active/; grep complete.md for ^## <candidate>$ shipped-check → scan complete/** (or generated index).
   • intake/intake.md — reconcile cross-ref against complete.md/issued/ and "retire to issued/ by hand" → new paths.
   • WORKFLOW.md, PyAutoMind/skills/OWNERSHIP.md — schema/pointer prose.
4. Mind scripts:
   • spawn.py — glob ("issued/*","DROP") → ("active/*","DROP") + ("complete/**", …); adjust ("complete.md","EMPTY") for the split/retired file.
   • status.sh — LIFECYCLE_DIRS (issued→active, add complete); complete_count counts complete/**/*.md not complete.md H2s.
   • prompt_sync.sh — grep -v '^issued/' → active/.
5. Smoke — throwaway prompt through create_issue → active/, simulate ship → complete/YYYY/MM/, run lifecycle.py --check clean.

Implementation Steps (PR-B)

6. Migration — lifecycle.py batch pass emits a review manifest (each issued/ file → target state + confidence); human/agent review; execute git mv + complete.md split in a committed series. Orphans → complete/unknown/, never dropped.
7. Template parity — spawn.py --write regenerate PyAutoMind-template; confirm spawn --check / spawn_drift CI green; republish.

Key Files

• PyAutoMind/scripts/lifecycle.py — new; the move/split/check engine.
• PyAutoMind/scripts/{spawn.py,status.sh,prompt_sync.sh} — path rewiring.
• PyAutoBrain/skills/{create_issue,ship_library,ship_workspace,start_dev,intake}/*.md — lifecycle path rewiring.
• PyAutoMind/complete.md (640KB, ~6000 lines) — split source.
• PyAutoMind/issued/ (444 files) — migration source.

Traps

• active.md - Repo lines are claims (feedback_active_md_dash_repos) — migration script must not misread them.
• Filenames map only fuzzily to complete.md slugs — decide state by ledger cross-ref, never by name.
• No JAX / behaviour change here; Mind-internal, so no downstream library API impact — but there IS a create_issue/ship_* behaviour change, hence the round-trip smoke.

Original Prompt

Click to expand starting prompt

See PyAutoMind/issued/lifecycle_state_split.md (moved from feature/pyautomind/ on issue creation). Original user request:

pyautomind/issued is huge and hard ot track, can we: (i) put all things not in issued in a folder called active, to make state split; (ii) make a complete folder which is for complete and separate from issued which are on going; (iii) put a folder for not yet active, e.g. drafts or thigns which are intaken but not start_dev; (iv) For complete, put them in folders by year and month in a way that appears numerically in order. As a follow up, I think we probbaly want to then build a wiki or indexing scheme for the complete folder so its really easy for an agent to look up old issues in a token light way, analogous to the wiki features used in autolens_assistant (e.g. RAG is dead).

[Jammy2211]

Checkpoint — engine built, mass-move blocked on a docs/ ambiguity

Autonomy: --auto supervised (feature cap at Difficulty large). Per checkpoint-and-continue I've built the independent, non-breaking part and I'm parking on the first real judgment fork before any large structural churn.

Done (committed locally on feature/lifecycle-state-split, not pushed)

• scripts/lifecycle.py — stdlib-only lifecycle engine:
  • move <name> [--date] — active/<name>.md → complete/<YYYY>/<MM>/ (month zero-padded → numerical order); date from --date or the matching complete.md - completed: line. This is the hook ship_library/ship_workspace will call.
  • split-complete — explode the 6000-line complete.md into one rich record per task under complete/YYYY/MM/<slug>.md (+ appends the original prompt when found). Dry-run validated on live data: 554 entries parse, 542 dated, 12 → complete/unknown/.
  • migrate — classify the legacy issued/→active/ files as still-active vs complete, ledger-backed only (fuzzy filename↔slug matches are listed UNSURE for human review, never auto-moved).
  • check — drift guard (mirrors repos_sync --check): no slug in both active.md and complete.md; every complete/ record has a complete.md entry; no file in two states. Green on current ledgers.
  • Filenames sanitised (safe_name) so slugs with spaces///em-dashes round-trip cleanly.

Blocked — Q1: how to handle docs/?

docs/ isn't a clean work-type dir. It mixes:

• docs-work-type prompts (Type: docs headers) — e.g. docs/workspaces/preloads_advanced_workspace_guide.md,
• reference specs that other code points at by path — notably docs/pyautobrain/spawn_spec.md, which spawn.py implements ("Implements docs/pyautobrain/spawn_spec.md").

Wrapping docs/ wholesale into draft/docs/ would misfile the specs and break that path reference. Options:

1. (Recommended) Move only the true Type:-headed prompts into draft/docs/; leave docs/pyautobrain/, docs/libraries/ etc. reference material in a docs/ that's demoted from "work-type" to a plain reference dir. Fix spawn.py's docs/* rule + the spawn_spec.md reference accordingly.
2. Leave docs/ entirely out of the draft/ wrapper — docs-work-type prompts keep living at docs/<target>/ (draft/ then wraps only the other 7 work-types). Least churn, but draft/ isn't a complete state split.
3. Move everything incl. specs into draft/docs/ and update every path reference to the specs. Cleanest tree, most breakage risk.

Also needs a nod — Q2: execute the structural churn + template republish?

Everything remaining is coupled to Q1, so it goes in one coherent commit once you answer:

• git mv issued/ → active/ (444 files, wholesale rename — the intelligent sort into complete/ is the separate PR-B), git mv <work-types>/ → draft/ (~88), create complete/ + complete/AGENTS.md.
• Rewire create_issue (→active/), ship_library/ship_workspace (call lifecycle move), start_dev/reference (active/ lookup), intake reconcile; status.sh, prompt_sync.sh, spawn.py rules; README/AGENTS taxonomy.
• spawn.py rule change drifts the PyAutoMind-template, so PR-A must also republish the template (an external-repo push). Confirm you want that done autonomously, or whether you'd rather do the republish yourself at merge.

Q3 — confirm the PR seam

PR-A = structure + rewire (above). PR-B = run lifecycle.py migrate/split-complete to sort active/ into complete/YYYY/MM/ + split complete.md, behind a review manifest. Phase-2 wiki (complete_archive_wiki.md) stays parked. OK?

On your answer to Q1 (+ Q2/Q3) I'll finish PR-A in one pass. Parking lifecycle-state-split as awaiting-input.

[Jammy2211]

PR-A complete — parked at supervised ship sign-off

docs/ fork resolved per your call (only Type:-headed prompts → draft/docs/; docs/pyautobrain/spawn_spec.md + other reference material stay in docs/). PR-A is implemented, validated, and committed on feature/lifecycle-state-split in both repos (local, not pushed — supervised convention).

What PR-A does

• Structure (git mv, history preserved): issued/ → active/ (440 files, wholesale — the intelligent sort into complete/ is PR-B); work-type dirs → draft/; new complete/ + complete/AGENTS.md (archive schema).
• Engine: scripts/lifecycle.py — move (active→complete/YYYY/MM by completion date), split-complete, migrate, check.
• Rewire: spawn.py rules (+spawn_spec.md), status.sh, prompt_sync.sh; skills create_issue (→active/), ship_library/ship_workspace (call lifecycle move on merge), start_dev, intake, WORKFLOW.md; docs AGENTS.md/README.md/REFERENCE.md/ROUTING.md.

Ship gate

Leg	Status
Tests	✅ PyAutoBrain pytest 46 passed. PyAutoMind has no suite → evidenced by lifecycle.py check OK, spawn.py dry-run unmatched: none (exit 0), status.sh clean, and a draft→active→complete/2026/07 round-trip smoke (throwaway removed).
Smoke	n/a — Mind/Brain are organism/doc repos, no downstream script surface.
Review	pending — a reviewer should git diff -M (≈535 of ~543 changes are pure renames; the substantive diff is ~14 files).
Heart YELLOW	not acknowledged at launch → ship parks per contract.

Needs your go-ahead

1. Push branches + open the PR-A PR? (supervised parks before PR-open; merge stays yours.)
2. Post-merge, in order: (a) republish the PyAutoMind-template (spawn --write + push — deferred deliberately so the template tracks main, not an unmerged branch); (b) then PR-B = lifecycle.py migrate/split-complete to sort active/ into complete/YYYY/MM/ + split complete.md, behind a review manifest.

Parking lifecycle-state-split as awaiting-input at ship sign-off.

[Jammy2211]

PR-A open (parked at your merge decision — merge stays human):

• PyAutoMind#72 — structure + engine + rewire (440 renames + lifecycle.py + spawn/status/sync/docs)
• Wire dev workflow to draft/ active/ complete/ lifecycle (PR-A, #71) PyAutoBrain#109 — skills half (create_issue/ship_*/start_dev/intake)

Merge Mind#72 first (structure), then Brain#109. Post-merge: republish the PyAutoMind-template (spawn_drift is expected red until then), then I'll run PR-B (lifecycle.py migrate/split-complete) behind a review manifest.

[Jammy2211]

PR-B prep — migration manifest (dry-run, read-only)

Ran lifecycle.py migrate/split-complete on the PR-A branch so the manifest is ready when you merge. PR-B execution stays blocked on Mind#72 merge (it operates on main's post-merge structure).

split-complete — 554 complete.md entries → one rich record each:
2026/04 144 · 2026/05 212 · 2026/06 28 · 2026/07 158 · unknown 13.

migrate — 440 active/ (legacy issued/) files:

• 3 stay active (ledger-backed, still in active.md)
• 146 → complete/YYYY/MM/ (ledger-backed — filename stem == a complete.md slug)
• 291 UNSURE (filename matched no slug — needs human review)

Design decision for PR-B

split-complete already folds the original prompt into each rich record (it looks up the matching active/ file and appends it under ## Original prompt). So for a done task, the active/ prompt file is redundant with its record. That reframes migrate:

• Recommendation: split-complete owns the record (+folded prompt) for every complete.md entry; migrate then only needs to resolve the active/ files left over — the 3 genuinely-still-active stay, and the rest (no live active.md entry) are either already-folded (drop) or abandoned (→ complete/unknown/ for review). The 291 UNSURE are the real manual-review surface — most are done under a differently-named slug.
• Alternative: keep prompt files separate from records (don't fold), and migrate moves all 437 done prompts into complete/ alongside their records. More files, cleaner separation.

I'll execute PR-B behind a full per-file manifest for your review once #72 is merged. Which framing do you prefer for the fold vs. separate question?

[Jammy2211]

PR-A merged + templates republished + PR-B open.

• PR-A — Mind#72 + Brain#109 MERGED (squash). New structure live on main; issued/ gone.
• Templates republished — spawn.py dry-run canary clean / unmatched none; PyAutoMind-template + PyAutoMemory-template force-synced (spawn_drift should go green next run).
• PR-B open — Mind#73 (your approved drop/fold framing): 557 dated records under complete/YYYY/MM/ + dropped the 147 provably-done active/ originals (active/ 440→293); lifecycle.py check green. Merge stays yours.

Two follow-ups I need your call on (both deliberately out of PR-B):

1. The 293 remaining active/ prompts — the un-auto-sortable legacy pile (only 3 match a live active.md task; the rest are old numbered prompts with no reliable record linkage). Bulk-drop (git-recoverable) / move to complete/_legacy/ / leave for gradual triage?
2. Retire complete.md — it's kept as the ledger backstop; fully retiring it means converting ship_* to write the completion record straight into complete/YYYY/MM/ instead of appending complete.md. Small, own PR.

Phase-2 (complete/index.md wiki) stays parked until these settle.