refactor: remove the NSS sampler and its cross-repo infrastructure

[Jammy2211]

Overview

NSS (the JAX-native Nested Slice Sampler, af.NSS) was integrated as a first-class sampler, but it dragged a bespoke install/CI/build apparatus in with it: a [nss] extra whose real dependencies (handley-lab/blackjax fork + yallup/nss) can't be declared as git+ direct URLs in an uploaded wheel, a manual post-install step, a dedicated unittest_nss job and test-ignore in the release workflow, a Heart CI-status fixture, and a Required-by: nss arviz footgun. Measured performance never justified that complexity (mixed at best — faster per-eval on MGE but OOM-prone on pixelization/Delaunay via vmap fan-out), and the sampler can return later as a genuine PyPI package. This task removes NSS and all of its supporting infrastructure across six repos.

Plan

• Delete the af.NSS sampler module, its tests, its [nss] extra, and the AGENTS.md footgun note from PyAutoFit (library root — this removes a public API).
• Strip the NSS tutorial section from the autofit_workspace searches/nest example (script + regenerated notebook).
• Remove NSS special-casing from PyAutoBuild's release workflow (the git+ direct-URL footgun guard, the separate unittest_nss job, the --ignore=…/nss test skip).
• Remove NSS references from PyAutoHeart (the "nss install smoke" CI-status fixture and the release_validation doc).
• Retire the NSS profiling runs in autolens_profiling (delete the searches/nss/ subtree, de-register it from the sweep/samplers), keeping the historical results/notes as evidence.
• Annotate the PyAutoMind nss_first_class_sampler epic as retired and void its outstanding issued/nss_* follow-up prompts.
• Phase into per-repo PRs, library-first (PyAutoFit merges before the workspace).

Detailed implementation plan

Affected Repositories

• PyAutoFit (primary)
• autofit_workspace
• PyAutoBuild
• PyAutoHeart
• autolens_profiling
• PyAutoMind

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoFit	main	clean
./autofit_workspace	main	dirty (231 — build outputs; isolated by worktree)
./PyAutoBuild	main	clean
./PyAutoHeart	main	clean
./autolens_profiling	main	dirty (14 — profiling outputs; isolated by worktree)
./PyAutoMind	tmp-land-navigator	dirty (1 — this prompt)

Suggested branch: refactor/remove-nss-sampler

Implementation Steps

Phase 1 — PyAutoFit (library; gates the rest):

1. git rm -r autofit/non_linear/search/nest/nss/ and test_autofit/non_linear/search/nest/nss/.
2. autofit/__init__.py — remove from .non_linear.search.nest.nss.search import NSS.
3. pyproject.toml — remove the [nss] comment block + nss = ["fastprogress<1.1"] extra (~lines 78–106). KEEP blackjax>=1.2.0 in [optional] (the NUTS search uses it).
4. AGENTS.md — remove the [nss] footgun bullet; drop "NSS" from the nested-sampler subsystem list.
5. Tests: full test_autofit/; because af.NSS is a public-API removal, leg-1 also runs PyAutoGalaxy + PyAutoLens suites (neither references NSS — expected green).

Phase 2 — autofit_workspace (user-facing tutorial; after Phase 1):

1. scripts/searches/nest.py — remove the NSS intro paragraphs and the Search: NSS section (~lines 10–44 intro refs + 300–391); confirm the Dynesty/Nautilus examples still run standalone.
2. Regenerate notebooks/searches/nest.ipynb from the edited script.
3. Smoke: run the trimmed nest.py.

Phase 3 — PyAutoBuild (release CI):

1. .github/workflows/release.yml — remove the [nss] git+ direct-URL footgun guard (~130–165), the separate unittest_nss job/handling, and the --ignore=test_autofit/non_linear/search/nest/nss skip (~250–256). No test dir → gate leg-1 n/a.

Phase 4 — PyAutoHeart:

1. tests/test_ci_status.py — remove the "nss install smoke" fixture entry; adjust assertions.
2. docs/release_validation.md — remove the [nss] wheel-blind-spot reference.
3. Tests: PyAutoHeart suite.

Phase 5 — autolens_profiling (dev repo; doc-light):

1. git rm -r searches/nss/ (9 files).
2. searches/sweep.py — remove the 9 ("nss", …) tuples.
3. searches/_samplers.py — remove the NSS-specific defaults; _setup.py/_runner.py — remove nss dispatch.
4. Clean nss mentions in README.md, hpc/README.md, skills/profile_likelihood/reference.md. Keep results/notes (historical A100 evidence).

Phase 6 — PyAutoMind (registry housekeeping):

1. Annotate z_features/complete/nss_first_class_sampler.md as retired/superseded.
2. Mark the ~8 issued/nss_* prompts void (moot once NSS is gone).

Testing approach

Per-repo suites (PyAutoFit + downstream libs, PyAutoHeart), workspace smoke for nest.py, and a final grep -ri nss sweep across all six repos to confirm no dangling af.NSS references remain.

Key Files

• PyAutoFit/autofit/non_linear/search/nest/nss/ — the sampler module (deleted)
• PyAutoFit/autofit/__init__.py — af.NSS export (removed)
• PyAutoFit/pyproject.toml — [nss] extra + git+ notes (removed)
• PyAutoBuild/.github/workflows/release.yml — NSS install/test special-casing (removed)
• PyAutoHeart/tests/test_ci_status.py, PyAutoHeart/docs/release_validation.md
• autofit_workspace/scripts/searches/nest.py — NSS tutorial section (removed)
• autolens_profiling/searches/nss/, searches/sweep.py, searches/_samplers.py

Autonomy

--auto, effective level SUPERVISED (refactor cap safe ∧ header supervised; a 6-repo public-API removal). Per-repo ship sign-offs are batched into a consolidated question on this issue; the run ends short of PR-open and merge stays human.

Original Prompt

Click to expand starting prompt

Refactor: remove the NSS nested sampler from PyAutoFit and retire all its supporting infrastructure across the organism. Delete the autofit/non_linear/search/nest/nss module, the af.NSS export, the [nss] git+ direct-URL extra in pyproject.toml, and its tests. Strip the NSS handling out of PyAutoBuild's release.yml (the git+ footgun guard, the separate unittest_nss job, the nss test-dir ignore) and remove the NSS references from PyAutoHeart (the nss install smoke CI fixture and release_validation docs). Sweep the autofit/autolens workspace examples and tutorials, retire the autolens_profiling NSS runs, and close out the PyAutoMind nss_first_class_sampler epic and its issued prompts. NSS performance never warranted this infrastructure cost and it can return as a genuine pip install later; we no longer support it.

[Jammy2211]

🅿️ Ship sign-off needed — --auto (supervised) parked at the ship gate

The NSS removal is implemented, tested, and committed locally (unpushed) on feature/remove-nss-sampler. Under the supervised autonomy level the ship sign-off is a human checkpoint, so the run stops here rather than pushing branches or opening PRs. Nothing is pushed; nothing is merged.

What was done (this run — the unclaimed core, Phases 1–2 + a stash)

Repo	Commit	Change
PyAutoFit	dbed2df	Delete the nss/ module + tests, the af.NSS export, the [nss] extra + git+ comment block, AGENTS.md notes. 1393-line deletion, public-API removal. blackjax kept (BlackJAXNUTS uses it).
autofit_workspace	4388124	Remove the Search: NSS tutorial section from scripts/searches/nest.py + matching notebooks/searches/nest.ipynb (22→20 cells).
autofit_workspace_developer	634995f	Preserve the NSS implementation in searches/nss/ (alongside ultranest/pyswarms) — module relativized as a drop-in, tests verbatim, README with the pinned blackjax/nss install recipe + re-mainline checklist, runnable example.py. (Human-directed scope addition.)

Four-leg gate results

1. Tests — test_autofit/: 1471 passed, 1 skipped, 0 failed (97s). Downstream (autogalaxy, autolens): grep-clean of any NSS reference and both import cleanly against the NSS-removed autofit; af.NSS confirmed gone. Full downstream suites not run — this is a deletion of an isolated sampler with zero downstream references; happy to run them before merge if you want.
2. Smoke — nest.py parses; notebook is nbformat-valid. Only the NSS block was removed; the Dynesty/Nautilus sections are unchanged from main, so no running code changed. Full triple-sampler execution not run (minutes-long, no behavioural change).
3. Review — CLEAN (review faculty surface across all 3 repos; pure removal + verbatim stash).
4. Heart — YELLOW, but every reason is pre-existing ambient organism state (stale 2026-07-09 workspace validation, 58 parked scripts, autolens_assistant version skew, repos.yaml drift, an 11-day-old PyAutoMind PR, install/release validation not run) — none NSS-related. Not acknowledged autonomously; needs your Heart-YELLOW ack to ship.

Deferred — Phases 3–6 are blocked (worktree conflicts)

These repos are held by other in-flight tasks, so they're registered in planned.md as remove-nss-sampler-remainder, to resume when the claims release:

• PyAutoBuild (release.yml [nss] git+ guard, unittest_nss job, --ignore .../nss) — blocked by codex-organ-skill-wrappers
• PyAutoHeart (test_ci_status.py nss fixture, release_validation.md) — blocked by codex-organ-skill-wrappers
• autolens_profiling (searches/nss/ subtree + sweep/samplers de-register) — blocked by preopt-breakdown-dashboard + profiling-preopt-campaign
• PyAutoMind (retire the nss_first_class_sampler epic + void issued/nss_*) — blocked by codex-organ-skill-wrappers

Landing Phase 1 first is safe: a release.yml --ignore of a now-absent dir is a harmless no-op, the Heart fixture is synthetic, and autolens_profiling is a dev repo outside CI.

On your approval I will

1. Acknowledge the Heart-YELLOW reason set above (if you confirm it).
2. Push the three feature/remove-nss-sampler branches and open pending-release PRs, library-first: PyAutoFit → autofit_workspace (behind the library-first merge gate) + autofit_workspace_developer (independent).
3. Leave merge to you (always human).

Reply here to approve (and ack Heart YELLOW), or tell me to hold / adjust.

[Jammy2211]

✅ Shipped to PR-open — maintainer approved + Heart-YELLOW acked

Ship sign-off approved and the ambient Heart-YELLOW reason set acknowledged. Three branches pushed, pending-release PRs open. Merge stays with you.

PR	Repo	Role
#1357	PyAutoFit	Library — removes af.NSS + [nss] extra (merge first)
PyAutoLabs/autofit_workspace#88	autofit_workspace	Tutorial — nest.py/notebook NSS section (⛔ merge after #1357)
Jammy2211/autofit_workspace_developer#20	autofit_workspace_developer	Stash — preserves NSS for re-mainlining (independent)

Merge order: #1357 → #88 (library-first gate); #20 anytime.

Still outstanding (this issue stays open)

Phases 3–6 remain blocked by other tasks' worktree claims (planned.md → remove-nss-sampler-remainder): PyAutoBuild release.yml, PyAutoHeart fixtures, autolens_profiling runs, PyAutoMind epic retirement. They'll resume when those claims release. Landing #1357 first is safe (leftover refs degrade to no-ops).

[Jammy2211]

✅ Core merged (library-first)

• refactor: remove the NSS nested sampler and its [nss] extra #1357 — merged (library; af.NSS + [nss] extra gone, blackjax retained)
• docs: remove the NSS section from the searches/nest example autofit_workspace#88 — merged (tutorial)
• searches: preserve the NSS sampler for re-mainlining Jammy2211/autofit_workspace_developer#20 — merged (NSS stashed at searches/nss/ for re-mainlining)

Worktree + feature branches cleaned up. Core (Phases 1–2 + stash) recorded in complete.md.

This issue stays open for the blocked remainder (planned.md → remove-nss-sampler-remainder): PyAutoBuild release.yml, PyAutoHeart fixtures, autolens_profiling runs, PyAutoMind epic retirement — waiting on the codex-organ-skill-wrappers PR (and the two profiling tasks) to release those repos. af.NSS is already gone from the library, so the remainder is pure infra cleanup and can land whenever the claims free up.

[Jammy2211]

Phases 3, 4 & 6 shipped (organ unblocked)

`codex-organ-skill-wrappers` merged into main, freeing PyAutoBuild / PyAutoHeart / PyAutoMind. Continued the sweep:

PR / target	Phase	Change
PyAutoLabs/PyAutoBuild#146	3	release.yml: NSS pytest --ignore special-case → plain pytest; stale unittest_nss comment gone; kept the general direct-reference-URL guard (de-NSS'd)
PyAutoLabs/PyAutoHeart#60	4	test_ci_status.py "nss install smoke" fixture → generic name; release_validation.md [nss] example genericized
#1361	(straggler)	dropped the stale [nss] gotcha ref from copilot-instructions.md
PyAutoMind (inline on main)	6	retired the nss_first_class_sampler epic + banner-voided the 8 issued/nss_* prompts

Also cleaned earlier: PyAutoFit#1360 (orphaned nss_install_smoke.yml).

Gate: PyAutoBuild 113p, PyAutoHeart 15p (test_ci_status); release.yml YAML-valid; all three repos grep-clean of nss. Two judgment calls flagged in the PRs: I kept the general direct-reference-URL packaging guard and the wheel-install doc requirement (de-NSS'd rather than deleted — useful beyond NSS). Say if you'd rather they go entirely.

Only Phase 5 left

autolens_profiling (searches/nss/ subtree + sweep/samplers de-register) is still held by preopt-breakdown-dashboard + profiling-preopt-campaign. Tracked in planned.md → remove-nss-sampler-remainder (now Phase-5-only). Once those release, that's the last piece — and this issue closes.

[Jammy2211]

✅ NSS removal complete — every phase merged

Phase 5 (autolens_profiling) landed once the profiling tasks freed the repo, and a final workspace-wide sweep caught a last batch of leftover prototype scripts. All merged:

PR	What
PyAutoLabs/autolens_profiling#65	Phase 5 — searches/nss/ cells, sweep/samplers de-register, 3 A100 HPC submits, docs (26 tests pass)
Jammy2211/autofit_workspace_developer#21	dev-clean — leftover searches_minimal/nss_* prototypes
PyAutoLabs/autolens_workspace_developer#92	dev-clean — leftover searches_minimal/nss_* prototypes (3 were broken, imported the removed lib)

Full ledger (all merged): #1357, autofit_workspace#88, autofit_workspace_developer#20 (core+stash) · #1360 (orphan smoke workflow) · PyAutoBuild#146, PyAutoHeart#60, #1361 (Phase 3/4/straggler) · autolens_profiling#65 (Phase 5) · autofit_workspace_developer#21, autolens_workspace_developer#92 (dev-clean) · Phase 6 (epic + 8 issued prompts retired) inline on Mind main.

Final sweep: zero functional NSS anywhere. Intentional keeps: the preserved implementation in autofit_workspace_developer/searches/nss/ (with the pinned-SHA re-mainline recipe), the results/notes/ A100 validation data, and the general direct-reference-URL packaging guard + wheel-install doc (de-NSS'd, useful beyond NSS).

This issue is ready to close — leaving that to you.

[Jammy2211]

Closing — NSS removal complete across all repos (13 PRs merged), final workspace-wide sweep confirms zero functional NSS. The implementation is preserved in autofit_workspace_developer/searches/nss/ with a pinned-SHA re-mainline checklist for when nss ships on PyPI.