docs: correct the VRAM batch-size guidance + make batch_size explicit - #469
Merged
Conversation
The __VRAM Use__ section told users "VRAM does not scale with batch size for the persistent buffers, so if the analysis fits within VRAM for batch_size=1 you should be able to push the batch size up (e.g. to 50) to maximise GPU throughput without running out of memory." That is the advice that produces the failure it should prevent. Memory splits into a fixed part and a per-evaluation part, and which dominates is the whole question; for interferometer data under a gradient search the per-evaluation part dominates, because the batched value_and_grad carries the forward tape. A 48-start SDP.81 fit measured ~1.79 GB per start, wanted ~86 GB, and OOMed two nightly release runs - at almost exactly the batch size this text recommended - while a single start fit comfortably. Replaces it with the fixed-vs-per-start split, the measured number, a pointer to start_here.py's batch_size=4, and a note that print_vram_use profiles the likelihood only unless passed gradient=True (Nautilus, used in this script, takes no gradients, so the default is right here). Notebook regenerated: the generator copies each block verbatim and the markdown cell was byte-identical to the docstring before and after. Refs PyAutoLabs/PyAutoFit#1452. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0171voyyTrr91hJ3vU5AjeVz
The corrected __VRAM Use__ text names interferometer/start_here.py as the script that passes batch_size=4, which the navigator generator extracts as a cross-reference. Adds that entry so the catalogue matches the scripts again. Exactly the one line the Catalogue staleness check asked for; llms-full.txt does not carry this prose and is unchanged. Refs PyAutoLabs/PyAutoFit#1452. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0171voyyTrr91hJ3vU5AjeVz
A new static check in PyAutoHands flags any MultiStart* search built without an explicit batch_size, because defaulting to None is how both interferometer/start_here.py files inherited an unbounded 48-way vmap on the same day and OOMed two nightly release runs. These three sites keep their current behaviour exactly - None is already the default, so nothing changes at runtime. What changes is that the choice is now written down where a reviewer sees it, with a pointer to the knob if a reader hits a memory limit. Deliberately not given a bounded value: these are CCD imaging, group and 1D-toy likelihoods rather than interferometer ones, and picking a real number needs a measurement on hardware this could not make. The imaging start_here.py entry points are the ones most worth profiling later, since a user on a laptop GPU meets them first. Refs PyAutoLabs/PyAutoFit#1452. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0171voyyTrr91hJ3vU5AjeVz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up for PyAutoLabs/PyAutoFit#1452, found while diagnosing it. Two related changes.
1. The VRAM guidance was wrong
The
__VRAM Use__section ofscripts/interferometer/modeling.pytold users:That is the advice that produces the failure it should prevent, at almost exactly the batch size it recommends. Memory splits into a fixed part and a per-evaluation part, and which dominates is the whole question. For interferometer data under a gradient search the per-evaluation part dominates, because the batched
value_and_gradcarries the whole forward tape.The measured case: a 48-start SDP.81 fit needed ~1.79 GB per start — ~86 GB in total — and OOMed two nightly release runs, while a single start fitted comfortably.
Replaced with the fixed-vs-per-start split, the measured number, a pointer to
start_here.py'sbatch_size=4, and a note thatprint_vram_useprofiles the likelihood only unless passedgradient=True.Nautilus— the search this script uses — takes no gradients, so the default remains right here; the caveat is there so a reader doesn't carry the number to a gradient search.2. Three sites left
batch_sizeimplicitPyAutoLabs/PyAutoHands#228 added a static check for
MultiStart*searches built without an explicitbatch_size. It found three here:scripts/guides/modeling/searches.py:165scripts/imaging/start_here.py:313scripts/multi_galaxy/start_here.py:375Behaviour is unchanged —
Noneis already the default. What changes is that the choice is written down where a reviewer sees it. No bounded value was invented: these are CCD imaging, group and guide likelihoods, and picking a real number needs a measurement this change could not make.imaging/start_here.pyis the one most worth profiling later — a user-facing entry point atn_starts=48, met first by someone on a laptop GPU.Merge ordering
print_vram_use(..., gradient=True), which exists only there.Files
Four scripts (one is prose-only, three one line each); matching notebooks regenerated byte-preservingly and each re-verified as valid JSON.
workspace_index.jsongained the one cross-ref the navigator catalogue check asked for.scripts/check_sizes.shpasses; the checker reports the tree clean.Could not update
Nothing. I checked whether
autogalaxy_workspacecarried the same false VRAM claim — it does not; its passage is vaguer but not wrong, so it is deliberately untouched. Its own four implicit-batch_sizesites are PyAutoLabs/autogalaxy_workspace#205.