fix: un-flake interferometer Delaunay test-mode FitException (+ NNLS NaN root cause)

[Jammy2211]

Overview

autolens_workspace/scripts/interferometer/features/pixelization/delaunay.py intermittently hard-fails CI smoke: the same commit (dce04672) failed one py3.12 run (ValueError: Points cannot contain NaN from the Delaunay/qhull triangulation) and passed another ~1 min apart, with no library merge in between — a true nondeterministic flake. The masked underlying cause is a NaN/non-PD in the interferometer Delaunay path; analysis.py:175-182 wraps ANY numpy-path error as af.exc.FitException (PR#607 parity guard so the sampler resamples), but under PYAUTO_TEST_MODE=2 there is no sampler, so a single unlucky eval hard-fails the script. The entry is currently parked in autolens_workspace/config/build/no_run.yaml.

Plan

• Phase 1 — un-flake test-mode (PyAutoLens + autolens_workspace): make the TEST_MODE bypass tolerate a single-eval FitException the way a real sampler does (resample / sentinel low FOM, logged so it stays visible) instead of hard-failing. This un-flakes all FitException-prone pixelization scripts in test-mode without masking real breakage in a real fit. Then un-park the interferometer Delaunay no_run.yaml entry and add the script to smoke_tests.txt (imaging Delaunay was added in autolens_workspace#307).
• Phase 2 — fix the numerical producer (PyAutoArray, separate PR): the smoking gun is fnnls.py:134 (alpha = np.min(d[q] / (d[q] - s_chol[q]))) — 45/250 local draws emit divide-by-zero/invalid RuntimeWarnings, usually benign but a candidate source of the fatal NaN/non-PD on CI. Guard/repair at the producer (no silent guards — fix the math) and/or the traced-mesh NaN conditioning.

Detailed implementation plan

Affected Repositories

• PyAutoLens (primary — Phase 1 test-mode tolerance, interferometer/model/analysis.py)
• autolens_workspace (Phase 1 — un-park marker + smoke)
• PyAutoArray (Phase 2 — util/fnnls.py producer fix)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoLens	docs/citations-exhaustive (other session)	dirty
./PyAutoArray	main	clean
./autolens_workspace	main	clean

Suggested branch: interferometer-delaunay-flaky-fitexception

Reproduction evidence (2026-07-21)

• True flake, not a library race: Feature/multi image visualization #307's two CI runs (same commit dce04672) ~1 min apart, one failed (qhull NaN) one passed, no PyAutoArray/PyAutoLens main merge between them.
• Local repro hard: 250/250 random prior-instance fit_from draws clean on current main — CI-runner thread/BLAS-dependent.
• Producer: 45/250 draws emit RuntimeWarning divide-by-zero at fnnls.py:134. Confirm which solver the script's modeling uses (use_positive_only_solver=False vs default positive/fnnls path).

Phase 1 steps

1. PyAutoLens/autolens/interferometer/model/analysis.py:175-182 — in the TEST_MODE path, catch the single-eval FitException, log it, return a sentinel low figure-of-merit instead of propagating (mirror the real-sampler resample semantics). Check for a shared base-analysis test-mode hook so imaging/pixelization scripts benefit too.
2. autolens_workspace/config/build/no_run.yaml — remove the interferometer Delaunay NEEDS_FIX entry.
3. autolens_workspace/smoke_tests.txt — add interferometer/features/pixelization/delaunay.py.
4. Verify: run the script in the exact CI smoke env repeatedly; run library unit tests; smoke.

Key Files

• PyAutoLens/autolens/interferometer/model/analysis.py — test-mode FitException tolerance
• PyAutoArray/autoarray/util/fnnls.py — Phase 2 divide-by-zero producer
• autolens_workspace/config/build/no_run.yaml, smoke_tests.txt — un-park + guard

Testing

Library unit tests (numpy-only) for the test-mode tolerance path; repeated CI-env script runs to confirm the flake is gone; PyAutoHeart vitals GREEN/YELLOW before ship. Library-first: PyAutoLens PR merges before the autolens_workspace un-park.

Original Prompt

Click to expand

Filed as PyAutoMind/draft/bug/autoarray/interferometer_delaunay_intermittent_qhull_nan.md. Surfaced 2026-07-21 while un-parking the Delaunay cluster (autolens_workspace#307, follow-up to #300/#301). Two masked signatures (qhull NaN vertices + non-PD inversion), both wrapped as FitException by analysis.py:175-182. Refs (all closed): autolens_workspace#300/#307/#308/#309.

[Jammy2211]

Phase 1 library PR created: PyAutoLabs/PyAutoFit#1408 (pending-release).

Re-scoped from PyAutoLens to PyAutoFit after the full CI traceback showed the failure runs through autofit's _fit_bypass_test_mode (abstract_search.py), not PyAutoLens analysis — source_pix_2 (Hilbert adaptive mesh) → bypass single-eval → log_likelihood_function → Delaunay interpolator → scipy_delaunay NaN. The fix makes the TEST_MODE=2 bypass tolerate a single-eval FitException (keep the -1e99 sentinel, log the cause) — central, fixes all analyses; only FitException is caught so real errors still fail loudly. Complements PyAutoLens#639.

Remaining (this task):

• Phase 1b (autolens_workspace, library-first — after #1408 merges): un-park the interferometer Delaunay no_run.yaml entry + add to smoke_tests.txt.
• Phase 2 (PyAutoArray, separate): fix the NaN/non-PD producer (fnnls.py:134 divide-by-zero / degenerate Hilbert-mesh vertices) so the fit is correct, not merely tolerated.

[Jammy2211]

Phase 1a MERGED (PyAutoFit#1408) and Phase 1b PR opened (autolens_workspace#311): un-park the interferometer Delaunay + add to smoke, now safe on the merged fix. Local run_smoke.py = 11/11 passed (interferometer delaunay 25.5s). Library-first satisfied. Remaining: Phase 2 — fix the underlying NaN/non-PD producer (fnnls.py:134 / degenerate Hilbert-mesh vertices) so the fit is correct, not merely tolerated.

[Jammy2211]

✅ Phase 1 complete (both PRs merged)

• Phase 1a — PyAutoFit#1408 MERGED: TEST_MODE=2 bypass tolerates a single-eval FitException (keep -1.0e99 sentinel + log), mirroring fitness.py:256's resample-to-reject. +2 tests; CI green 3.12/3.13.
• Phase 1b — autolens_workspace#311 MERGED: un-parked the interferometer Delaunay no_run.yaml entry + added to smoke_tests.txt. CI green on 3.12 and 3.13 against the merged fix — the flake no longer hard-fails.

User-facing problem resolved: the intermittent CI failure is fixed; any future FitException-raising instance in test-mode is now tolerated (all analyses benefit, not just this script).

Remaining — Phase 2 (open, non-urgent)

Fix the underlying producer in PyAutoArray so the fit is correct, not merely tolerated: the NaN Delaunay-mesh vertices in source_pix_2 (Hilbert adaptive mesh — the actual failing stage) and/or the fnnls.py:134 divide-by-zero (45/250 local draws hit it). This is harder — the flake is CI-thread-dependent and not locally reproducible on the Overlay stage (250 draws clean), so it needs a source_pix_2 Hilbert-stage reproduction first. Keeping this issue open for Phase 2.