feat: add scaffolding to migrate subpackages to the new layout

[avinash2692]

Stacked on #63. This PR's diff includes the receiver-workflow
commits from #63 until that PR merges. Review can proceed in
parallel; merge order is #63 first, then this PR.

Foundation work for the contribs repo restructure: a root meta-package,
the cookiecutter template that generates new subpackages in their
target shape, the structural-validation gate, a new CI workflow that
scopes the matrix to PR-touched packages, and a daily smoke against
mellea@main with an auto-issue bot for the 21-day archival lifecycle.

No subpackages migrate in this PR; all six legacy subpackages keep
running through legacy-ci.yml until each one is ported in a later PR.

Landed in this PR

• Root meta-package + dev-only uv.lock (ruff, mypy, cookiecutter,
  pre-commit, pytest, PyGithub)
• Cookiecutter template that generates <subpkg>/mellea_contribs/<name>/<core mirror>/...
  with [tool.mellea-contribs.ci] defaults and core-path validation
• validate-structure script + workflow: required files/dirs,
  hatch wheel packages = ["mellea_contribs"], namespace package
  shape, explicit mellea version constraints, distribution-name
  uniqueness; legacy subpackages grandfathered
• New ci.yml with PR-scoped discovery (docs-only,
  cookiecutter-only, root, subpackage, union, stacked-PR), reusable
  package-ci.yml reading [tool.mellea-contribs.ci] flags
• Old ci.yml preserved as legacy-ci.yml, scoped to
  mellea_contribs/**
• Daily smoke-against-mellea-main.yml + .github/smoke-matrix.json
  gating file (starts empty; subpackages opt in as they migrate)
• Auto-issue bot (auto_issue_bot.py) with PyGithub real mode
  and an in-memory fake for tests; tracks the 2-consecutive-reds
  threshold, recovery comments, and the day-7 / 14 / 21
  archival-label timeline (14 unit tests)
• auto-issue-archival.yml workflow that runs the bot's archival
  pass once daily

Followups (separate PRs)

• Documentation: contribs release process (coordinated single-tag
  flow, per-package bump tier flow, straggler-exclusion policy,
  post-merge tag push that fires release.yml)
• Documentation: contributor "create your package" guide walking
  through the cookiecutter template end-to-end (command invocation,
  generated file tour, where implementation code goes, how to run
  tests locally, what CI runs on the resulting PR)

How to verify locally

# Cookiecutter dry-run (generates a working subpackage):
uv run --with cookiecutter cookiecutter ./cookiecutter \
  --no-input name=demo core_path=stdlib.sampling_algos
cd demo && uv sync && uv run pytest -v   # 1 PASS

# Foundation tests (51 total):
uv run pytest -c pyproject.toml --rootdir=. .github/scripts/ -v
# 14 validate-structure + 8 discover_subpackages + 14 auto-issue bot
# + 8 update_contribs_versions + 7 open_per_package_bump_prs

# validate-structure against the live repo (legacy paths grandfathered):
uv run python .github/scripts/validate_package_contract.py
# validate-structure: PASS - all subpackages conform.

[planetf1]

Eight findings inline — see comments. Critical issues around the bot state-branch write permissions and concurrent-write race; fix suggestions included throughout.

[planetf1]

Two related issues here.

Permission: the bot writes the state JSON back to the bot-state branch, so contents: read will 403 on every save — nothing ever persists.

Suggested change

	contents: read
	contents: write

Race condition: before doing that though, it's worth tackling why each leg writes independently. The smoke matrix runs legs in parallel and each one does a read-modify-write against the same JSON blob on the same branch. Concurrent runs either silently clobber each other's counter updates (last-write-wins), or hit a 409 SHA conflict that the except Exception: in _save_persistent_state swallows before trying create_file on an already-existing file — unhandled 422, step crashes.

The cleaner fix for both: have each leg emit its outcome as a job output or artifact, then run the bot once in a post-matrix aggregation job that sees all results together:

aggregate:
  needs: smoke
  if: always()
  permissions:
    contents: write
    issues: write

Smoke legs stay at contents: read, the bot only ever runs once per workflow invocation, no concurrent writes possible. Needs a small addition to the bot to accept bulk outcomes rather than a single package — but probably less work than making the per-leg writes race-safe.

[planetf1]

Same issue as smoke — the bot writes to the bot-state branch during apply-archival, so this needs contents: write too.

Suggested change

	contents: read
	contents: write

[planetf1]

record-failure only fires when pytest itself reports failure, but if the install step blows up first, pytest never runs — its outcome comes back as skipped and the condition is false. An install break against mellea@main silently goes unreported, which is arguably the most important thing to catch.

failure() alone covers both cases — it fires if any step in the job failed:

Suggested change

	if: failure() && steps.pytest.outcome == 'failure'
	if: failure()

[planetf1]

The except Exception: here catches everything — a transient 500, a rate-limit 403, a network timeout, malformed JSON — and in all those cases silently returns a fresh empty BotState. That then gets written back, wiping whatever was actually stored.

Same pattern in _save_persistent_state (line 449): a 409 SHA conflict hits the except, tries create_file on an already-existing path, gets a 422, and crashes with no useful message.

Both should only catch the genuine "file doesn't exist" case. Needs a module-scope from github import GithubException, then:

Suggested change

	except Exception: # pragma: no cover - first run / missing branch
	return BotState()
	except GithubException as e:
	if e.status != 404:
	raise
	return BotState()

[planetf1]

timeout_minutes and python_versions are both parsed from [tool.mellea-contribs.ci] and emitted as step outputs, but neither is actually consumed — the timeout is hard-coded here and there's no Python matrix. skip_ollama is wired up, so the gap is pretty visible.

The timeout one matters immediately: crewai_backend runs at 90 minutes in legacy CI and will get silently killed at 30 on migration.

Suggested change

	timeout-minutes: 30 # Default; subpackages override via [tool.mellea-contribs.ci]
	timeout-minutes: ${{ fromJson(steps.ci-flags.outputs.timeout_minutes) }}

python_versions is the same story but needs the job split into two to use a dynamic matrix — that's a reasonable follow-up PR before anyone migrates.

[planetf1]

This grep catches changes to ci.yml — but ci.yml is now the new workflow. A change to legacy-ci.yml itself (which does trigger this workflow) won't set WORKFLOW_CHANGED, so the "run all legacy packages" path never fires when you edit the legacy harness.

Suggested change

	WORKFLOW_CHANGED=$(git diff --name-only $BASE_REF HEAD | grep -E "\.github/workflows/(ci\.yml|quality-generic\.yml)" || true)
	WORKFLOW_CHANGED=$(git diff --name-only $BASE_REF HEAD | grep -E "\.github/workflows/(legacy-ci\.yml|quality-generic\.yml)" || true)

[planetf1]

Pipe-to-shell from an external origin with no checksum — if ollama.com ever serves a bad script (CDN issue, compromised account), every CI run executes it with your GITHUB_TOKEN in the environment. This is the same supply-chain vector as the tj-actions incident earlier this year.

Worth pinning to a specific release and verifying the SHA, or switching to a pinned container image.

[planetf1]

Same root cause as the _load_persistent_state issue above — except Exception: here catches a 409 SHA conflict (from the concurrent-write race) and falls through to create_file on an already-existing path, which raises an unhandled 422 and crashes the step. Same fix applies:

Suggested change

	except Exception: # pragma: no cover - first commit on the branch
	repo.create_file(
	path=STATE_FILE_PATH,
	message=commit_message,
	content=payload,
	branch=branch,
	)
	except GithubException as e:
	if e.status != 404:
	raise
	repo.create_file(
	path=STATE_FILE_PATH,
	message=commit_message,
	content=payload,
	branch=branch,
	)

[planetf1]

If the git diff fails (e.g. a shallow clone with a stale base SHA), this falls back to git ls-files — every tracked file in the repo. That trips the root trigger and silently promotes the PR to a full matrix run, with no indication anything went wrong.

Better to fail loudly so the cause is diagnosable:

Suggested change

	CHANGED=$(git diff --name-only "$BASE_SHA"...HEAD || git ls-files)
	CHANGED=$(git diff --name-only "$BASE_SHA"...HEAD)

[avinash2692]

Thanks @planetf1, pushed 5 commits.

The race was the interesting one. After tracing it, the legs were actually writing disjoint keys (consecutive_failures[package] only), so the conflict was entirely substrate-level: one JSON blob, one SHA, contention. Splitting storage to .github/bot-state/<package>.json matches the access pattern, so parallel legs never share a path.

Rest of the findings:

• contents: write on smoke + archival, narrowed except Exception
• if: failure() so install breaks against mellea@main actually report
• legacy-ci.yml grep was matching ci.yml (the new file); also dropped
  || git ls-files in ci.yml so a bad base SHA fails loud
• package-ci.yml: hoisted toml read into a read-config job so
  timeout-minutes actually consumes it; pinned Ollama to a release tarball

python_versions is parsed but unwired, left for a follow-up since matrix expansion is its own change. Bot tests still green (14 passed). Please take a look and let me know.