add a script to find and apply package updates#1159
Conversation
Add scripts/update_downloads.py, which finds updates to the entries
to pythonbuild/downloads.py. The optional --write argument will write
the necessary updates to the file.
Run using:
uv run scripts/update_downloads.py
|
This script was largely AI generated using Codex. The prompt that began the session was:
I have used this script to kick off many of the recent update including #1153, #1156, and #1157. I plan on testing it with the upcoming 3.15.0b3 release. |
|
Should have tagged this with This work could be expanded to include tests and an actions to run + apply changes on a regular schedule. That can be done later if needed. |
|
Example results: |
|
This worked well to start the 3.15.0b3 update: |
geofft
left a comment
geofft
left a comment
There was a problem hiding this comment.
This generally seems good to me, although I only applied the level of review I would to human-authored code.
Can you extend it to 1) push to the mirror with GITHUB_TOKEN if it needs to mirror something and 2) update wanted_version in the OpenSSL test case, as the comments on the OpenSSL downloads instruct (or do something equivalent, like have the test case import this file)?
| def artifact_metadata(client: HttpClient, url: str) -> tuple[int, str]: | ||
| digest = hashlib.sha256() | ||
| size = 0 | ||
| with client.open(url) as response: | ||
| while chunk := response.read(1024 * 1024): | ||
| size += len(chunk) | ||
| digest.update(chunk) | ||
| return size, digest.hexdigest() |
There was a problem hiding this comment.
Some upstreams have .sha256 files etc. that we can optionally use to make the process a little faster.
| return urllib.parse.urljoin(MIRROR_BASE_URL, filename) | ||
|
|
||
|
|
||
| def _line_offsets(source: str) -> list[int]: |
There was a problem hiding this comment.
This whole next section to handle modifying the Python source in place is clever, and I suppose harmless in the brave new world, but I would have been inclined to do something like
a) turn inline comments into actual data
b) ensure that the file is formatted in some predictable way right now, and regenerate the file in that same way (either by generating it that way in the first place, or running ruff after outputting, or whatever)
I think it also would be reasonable to turn the file into JSON or YAML or something instead of Python, if that's helpful, but I suppose ast is fine with Python.
2 participants
Add scripts/update_downloads.py, which finds updates to the entries in pythonbuild/downloads.py. The optional --write argument write any updates, including calculated metadata to the file.
Run using: