From 296b56b5841d668894789b810c5d52810cff1888 Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Tue, 4 Aug 2026 14:38:48 +0100 Subject: [PATCH] docs(search): state the opt-in rule for _test_mode_samples_info MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The hook's docstring told subclasses to override it 'so that tutorial scripts and downstream code can access those keys without KeyError', which reads as a per-sampler obligation — so the fact that only BlackJAXNUTS overrides it, out of nine searches that write samples_info, looks like an oversight in the other eight. It isn't. No library path reads these diagnostic keys under bypass: the properties that read them (SamplesMCMC.total_steps, SamplesNest.total_samples, ...) live on Samples subclasses the bypass never constructs, since _fit_bypass_test_mode always builds a SamplesPDF. The only consumers are workspace scripts reading samples_info[...] directly, and across all eleven workspace/tutorial repos there are exactly two: searches/mcmc.py prints NUTS diagnostics (bypassed on every PR — no __Env__ declaration, and it is in smoke_tests.txt), which is why #1260 added the hook and the NUTS override; and the multi-start auto-convergence assertion script, which asserts on total_steps and was fixed with ENV: real_search jax instead (autofit_workspace_test#83). Document the rule those two decisions actually follow — prints -> placeholders, asserts -> real search — and warn that adding placeholders for an asserting reader is worse than the KeyError it replaces, since the assert then silently passes on a stub value. Docstring only; no behaviour change. Closes #1448 Co-Authored-By: Claude Opus 5 --- autofit/non_linear/search/abstract_search.py | 33 +++++++++++++++++--- 1 file changed, 28 insertions(+), 5 deletions(-) diff --git a/autofit/non_linear/search/abstract_search.py b/autofit/non_linear/search/abstract_search.py index fcb9e45af..2005074c2 100644 --- a/autofit/non_linear/search/abstract_search.py +++ b/autofit/non_linear/search/abstract_search.py @@ -981,11 +981,34 @@ def _test_mode_samples_info(self) -> dict: Sampler-specific keys to merge into ``samples_info`` when the sampler is bypassed via ``PYAUTO_TEST_MODE=2`` or ``=3``. - Override in subclasses to add the diagnostic keys that the real - run would populate (e.g. NUTS ESS, MCMC autocorrelations) so that - tutorial scripts and downstream code can access those keys - without ``KeyError``. Use NaN/0 placeholders — the bypass did not - actually sample. + **Opt-in, not a per-sampler obligation.** Most searches do not + override this, and that is correct — no library code reads these + diagnostic keys under bypass. The properties that read them + (``SamplesMCMC.total_steps``, ``SamplesNest.total_samples``, …) + live on ``Samples`` subclasses the bypass never constructs; + ``_fit_bypass_test_mode`` always builds a ``SamplesPDF``. The only + consumers are workspace scripts reading ``samples_info[...]`` + directly, so override this only when such a script exists. + + Which fix applies depends on what that script does with the keys: + + - It **prints** them (a tutorial, whose point is the prose, so it + may legitimately run bypassed) → override here, returning NaN/0 + placeholders. The bypass did not sample; honest empties. + - It **asserts** on them (a test) → the script must not run + bypassed at all. Give it an ``ENV: real_search`` declaration + instead. Do **not** add placeholders for an asserting reader: + the assert then silently passes on a stub value, which is worse + than the ``KeyError`` it replaced. + + Both live cases follow that split. ``BlackJAXNUTS`` overrides this + because ``autofit_workspace/scripts/searches/mcmc.py`` prints + ``ess_min`` / ``n_divergent``, carries no ``__Env__`` declaration + and is in that workspace's ``smoke_tests.txt``, so it runs + bypassed on every PR (PyAutoFit #1260). + ``AbstractMultiStartGradient`` deliberately does not: its only + reader asserts on ``total_steps``, so that script declares + ``ENV: real_search jax`` instead (autofit_workspace_test #83). """ return {}