Skip to content

prompt: route covariance-interpolator-rng-seed (#1450) → start_library - #127

Merged
Jammy2211 merged 1 commit into
mainfrom
claude/covariance-interpolator-rng-seed-8zexp3
Aug 5, 2026
Merged

prompt: route covariance-interpolator-rng-seed (#1450) → start_library#127
Jammy2211 merged 1 commit into
mainfrom
claude/covariance-interpolator-rng-seed-8zexp3

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Mind state for the PyAutoFit interpolator flake — the Class B release blocker
that killed the 2026.8.2.1 live release.

Paired with PyAutoLabs/PyAutoFit#1451 (fix) and PyAutoLabs/PyAutoFit#1450 (issue).

Normally this would land on main via prompt_sync_push; this session is
branch-scoped, hence a PR.

What changed

  • draft/bug/autofit/covariance_interpolator_test_unseeded_rng.md
    active/
    , header advanced to Status: issued with the issue URL.
  • active.md — new covariance-interpolator-rng-seed entry.
  • draft/feature/autofit/search_seed_reproducibility.md — new follow-up
    prompt (see below).

python3 scripts/lifecycle.py check → OK.

Why the registry entry is long

It records that the prompt's premise was wrong, which matters more than the
fix. The prompt attributed the flake to the unseeded np.random.random() in the
test body. Measurement says otherwise:

  • Pre-fix failure rate 1.65% (33/2000, 95% CI 1.18–2.31%) — measured before
    touching anything, as the prompt's step 2 required.
  • Seeding only np.random leaves std at 1.95 of 2.05; the test still fails.
    The dominant source is dynesty's rstate (Generator(PCG64(None)), OS
    entropy), with the stdlib random module used by initializer.py third.
  • The tolerance is not mis-calibrated — a full search recovers
    25.0496 ± 0.0039, 500× tighter. The autouse limit_maxcall fixture is what
    turns the assertion into a coin flip.
  • The sibling test_single_variable was flaking worse (3.6%) with no random
    call of its own.
  • The LinAlgError guards are explicitly ruled out as a different root cause
    (rank-deficient fixture matrix, cond 1.3e17 — deterministic input).

The entry also records the known limitation the fix does not solve, so nobody
later reads "seeded" as "the assertion is now meaningful".

The follow-up prompt

search_seed_reproducibility.md captures the real gap the fix exposed:
PyAutoFit has no supported way to seed a search.
AbstractDynesty.search_kwargs is a closed dict, so rstate cannot be threaded
through even though dynesty accepts it — which is why the fix must monkeypatch
a third-party module from a test, and why users cannot reproduce a fit. It flags
the identifier/serialization trap (store the integer seed, not a Generator;
decide deliberately about __identifier_fields__).

Filed rather than folded in, to keep the release-unblocking change test-only.

🤖 Generated with Claude Code

https://claude.ai/code/session_016vdGab4C5rWumTCXkWHPyh


Generated by Claude Code

Issue PyAutoFit#1450 filed; prompt advanced draft/bug/autofit/ -> active/
and registered in active.md.

Records the finding that the prompt's premise was wrong: the unseeded
np.random.random() in the test body is not the dominant cause. The
dominant source is dynesty's rstate (Generator(PCG64(None)), OS entropy),
with the stdlib random module used by initializer.py as a third source.
Measured pre-fix failure rate 1.65% (33/2000, 95% CI 1.18-2.31%), and the
sibling test_single_variable - which has no random call at all - was
flaking worse at 3.6%.

Also files the follow-up the fix exposed: PyAutoFit has no supported way
to seed a search, because AbstractDynesty.search_kwargs is a closed dict
and rstate cannot be threaded through it. That is why the fix has to
monkeypatch a third-party module from a test, and why users cannot
reproduce a fit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016vdGab4C5rWumTCXkWHPyh
@Jammy2211
Jammy2211 merged commit acc4a7d into main Aug 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants