refactor to work with gardenlinux 0.10.21 by yeoldegrove · Pull Request #13 · gardenlinux/glrd

yeoldegrove · 2025-09-09T13:32:31Z

What this PR does / why we need it:

Code refactoring

glrd/git.py — replace gh/git CLI with python-gardenlinux-lib

Use gardenlinux.github.Client (PyGithub) and gardenlinux.git.Repository
(pygit2) instead of subprocess calls to gh and git. Both are already
pulled in transitively by the gardenlinux dependency.

get_github_releases: Client().get_repo(...).get_releases()
get_git_commit_from_tag: get_git_ref() with annotated-tag dereferencing
get_git_commit_at_time: Repository.checkout_repo() + pygit2 walk
cleanup_temp_repo: dereference _repo_instance before rmtree

Auth change: GITHUB_TOKEN is now required for GitHub API calls instead
of gh auth login. Remove gh CLI install from Containerfile.

release.py (renamed from models.py) — single source of truth

Harden Version for 'next' sentinel; make to_dict faithful
Wire as SSoT in query.py (filter, sort, latest, active/archived) and
manage.py (create, parse_release_name, ensure_isodate_and_timestamp)
Add Release.default_name() and Release.github_release_url()

manage.py — remove 547 lines of duplicates

Remove functions now imported from git.py, validation.py, and s3.py.
Remove unused imports. Net -547 lines (-29%).

schema.py — unify schema_v1.py and schema_v2.py

Generate SCHEMA_V1 and SCHEMA_V2 from shared builder functions.
Reduces ~580 lines of duplicated definitions. Delete schema_v1.py and
schema_v2.py.

Other refactors

Extract validation functions to glrd/validation.py
Extract S3 operations to glrd/s3.py
Extract shared helpers into glrd/util.py (V2_SCHEMA_THRESHOLD,
split_releases_by_type, run_subprocess, parse_isodatetime, fatal_error,
resolve_flavors, merge_input_data)
Bump python-gardenlinux-lib dependency from 0.6.0 to 0.10.21.
Fix circular import: move merge_input_data from manage to util
Fix filter_releases crash on unknown --type values
Add --no-flavors flag and GLRD_SKIP_FLAVORS env var
Add --input-type/--input-url/--input-file-prefix/--query-input-format
to glrd-manage for offline/custom-source queries
Fix deprecated datetime.utcfromtimestamp() usage throughout

Tests

Make full test suite fully offline (was hanging on network; now 97
tests in ~40 s) via autouse conftest fixture
Add tests/test_git.py: 14 unit tests for all git.py functions, mocking
gardenlinux.github.Client and using a local pygit2 bare-repo fixture
Add tests/test_release_model.py: 21 unit tests for the release model
Add 6 offline CLI integration tests (active/archived/latest/version
filter, no-flavors, unknown-type guard)

Documentation

Replace monolithic docs/overview/README.md with a full Diataxis
structure: reference/index.md, reference/cli.md, reference/schema.md,
explanation/release-lifecycle.md, how-to/index.md, how-to/run-glrd.md,
how-to/query-releases.md, how-to/manage-releases.md
Replace gh CLI prerequisite in run-glrd.md with GITHUB_TOKEN
prerequisite; add GITHUB_TOKEN callout to manage-releases.md

… schema.py Reduces ~580 lines of duplicated schema definitions by generating both SCHEMA_V1 and SCHEMA_V2 from shared builder functions. The only difference between v1 and v2 is that v2 requires a 'patch' field in version objects for minor, nightly, and dev release types. - Create glrd/schema.py with factory functions for generating schemas - Update manage.py to import from glrd.schema instead of the old files - All schema validation tests pass (7/7)

- Add missing ERROR_CODES entries: output_error, s3_output_error, input_parameter_missing, input_parameter_error, subprocess_output_missing - Add fatal_error() utility function for centralized error handling - Fix deprecated datetime.utcfromtimestamp() usage in util.py and manage.py (use datetime.fromtimestamp with timezone.utc instead) - Add timezone import to both util.py and manage.py

Add commonly used utility functions: - V2_SCHEMA_THRESHOLD constant (2017) for v1/v2 schema decisions - uses_patch_version() helper function - split_releases_by_type() to split releases by type into a dict - run_subprocess() for standardized subprocess execution - parse_isodatetime() for consistent datetime parsing with error handling All schema tests pass.

Replace duplicated release type filtering logic (~15 lines repeated 3 times) with calls to the shared split_releases_by_type() utility function from util.py. This eliminates duplicate code and ensures consistent behavior.

- Remove unused import in schema.py - Fix line length issues in schema.py - Add proper blank line after function in util.py - Add proper type hints for parse_isodatetime - Format all modified files with black

Phase 1 - Dataclasses: - Add glrd/models.py with ReleaseType enum, Version, LifecyclePhase, Lifecycle, GitInfo, Release, ReleaseCollection, parse_release_name - Centralizes the 'major >= 2017' version threshold logic in Version class Phase 2 - S3 module: - Extract S3 operations to glrd/s3.py - Import s3 functions in manage.py for backward compatibility - Functions: create_s3_bucket, upload_to_s3, download_from_s3, merge_existing_s3_data, download_all_s3_files, upload_all_local_files All schema validation tests pass.

Phase 2 - Git module: - Extract Git operations from manage.py to glrd/git.py - Functions: get_github_releases, get_git_commit_from_tag, get_git_commit_at_time, get_garden_version_for_date, create_initial_releases, create_initial_nightly_releases, cleanup_temp_repo - Fix date increment logic in create_initial_nightly_releases (use timedelta) - Import git functions in manage.py All schema tests pass.

Phase 2 - Validation module: - Extract validation functions from manage.py to glrd/validation.py - Functions: validate_input_version_format, get_schema_for_release, validate_release_data, validate_all_releases - Use V2_SCHEMA_THRESHOLD constant instead of hardcoded 2017 - Import validation functions in manage.py All schema tests pass.

- Remove unused datetime import from models.py - Remove unused major_version variable in git.py - Fix trailing whitespace in git.py - Format code with black All tests passing.

- Change from 'deepdiff import DeepDiff' to 'deepdiff.diff import DeepDiff' to fix lint warning about incorrect module import Note: manage.py still contains duplicate function definitions that are imported from new modules (s3.py, git.py, validation.py). These duplicates are functionally harmless as Python uses the imported versions, but they trigger F811 lint warnings. Removing them automatically breaks the file due to complex interdependencies. This is acceptable technical debt that can be addressed incrementally. All 29 tests passing.

Clean up manage.py by removing 522 lines of duplicate function definitions that are now imported from other modules: Duplicate functions removed: - validate_input_version_format, get_schema_for_release (from validation.py) - cleanup_temp_repo, get_github_releases, get_git_commit_from_tag, get_git_commit_at_time, get_garden_version_for_date, create_initial_releases, create_initial_nightly_releases (from git.py) - validate_release_data, validate_all_releases (from validation.py) - download_all_s3_files, upload_all_local_files (from s3.py) Unused imports removed: - fnmatch, shutil, subprocess, timedelta, timezone - jsonschema.validate, jsonschema.ValidationError - SCHEMA_V1, SCHEMA_V2 - extract_version_data, NoAliasDumper - save_output_file alias File size reduced from 1,911 to 1,364 lines (-547 lines, -29%) Format with black to fix spacing issues. All 29 tests passing. Lint clean (no errors or warnings).

…test suite fully offline - Bump python-gardenlinux-lib dependency from 0.10.20 to 0.10.21 - Rename models.py -> release.py; harden Version for 'next' sentinel; make to_dict faithful (2135 real releases round-trip with 0 mismatches) - Wire release.py as single source of truth in query.py (filter, sort, latest, active/archived) and manage.py (create builds Release object, parse_release_name delegates to model, ensure_isodate_and_timestamp delegates to LifecyclePhase.ensure_complete) - Add Release.default_name() and Release.github_release_url() so name and URL generation are centralized in the model - Fix circular import: move merge_input_data from manage to util - Remove duplicate S3 functions from manage.py (save_output_file, create_s3_bucket, upload_to_s3, download_from_s3, merge_existing_s3_data); import from glrd.s3 throughout - Delete unused schema_v1.py and schema_v2.py - Wire previously dead --s3-create-bucket flag in handle_releases - Fix filter_releases crash on unknown --type values (restore prior behavior of silently ignoring unknown types instead of raising ValueError) - Fix dead pass branch in diff_releases to log added/removed items - Add resolve_flavors() to util.py; add --no-flavors flag and GLRD_SKIP_FLAVORS env var to skip network-dependent flavor lookup - Skip git commit lookup for next/major release types (they never store it) - Add --input-type/--input-url/--input-file-prefix/--query-input-format to glrd-manage so update/delete queries can run offline or against custom sources - Make full test suite offline (was hanging indefinitely; now 83 tests in ~39s): autouse conftest fixture, offline query injection in test helpers - Add 22 new tests: 21 unit tests for release model + 6 offline CLI integration tests (active/archived/latest/version filter, no-flavors, unknown-type guard) - Fix test_schema_validation.py import (glrd.validation, not glrd.manage) - Update README with current --help output and offline testing guidance - Net -375 lines; flake8 + black clean Signed-off-by: Eike Waldt <waldt@b1-systems.de> On-behalf-of: SAP <eike.waldt@sap.com> Assisted-by: Kilo:claude-opus-4-8 Assisted-by: Kilo:claude-sonnet-4-6

Replace the monolithic docs/overview/README.md with a proper Diataxis structure: - docs/reference/index.md → hub: reference/supporting_tools/glrd/index.md - docs/reference/cli.md → hub: reference/supporting_tools/glrd/cli.md - docs/reference/schema.md → hub: reference/supporting_tools/glrd/schema.md - docs/explanation/release-lifecycle.md → hub: explanation/glrd-release-lifecycle.md - docs/how-to/index.md → hub: how-to/releases/glrd/index.md - docs/how-to/run-glrd.md → hub: how-to/releases/glrd/run-glrd.md - docs/how-to/query-releases.md → hub: how-to/releases/glrd/query-releases.md - docs/how-to/manage-releases.md → hub: how-to/releases/glrd/manage-releases.md Signed-off-by: Eike Waldt <waldt@b1-systems.de> On-behalf-of: SAP <eike.waldt@sap.com> Assisted-by: Kilo:claude-opus-4-8 Assisted-by: Kilo:claude-sonnet-4-6

Use gardenlinux.github.Client (PyGithub) and gardenlinux.git.Repository (pygit2) instead of subprocess calls to gh and git. Both are already pulled in transitively by the gardenlinux dependency. - get_github_releases: Client().get_repo(...).get_releases() - get_git_commit_from_tag: get_git_ref() with annotated-tag dereferencing - get_git_commit_at_time: Repository.checkout_repo() + pygit2 walk - cleanup_temp_repo: dereference _repo_instance before rmtree Auth change: GITHUB_TOKEN is now required for GitHub API calls instead of gh auth login. Add GITHUB_TOKEN prerequisites to run-glrd.md and manage-releases.md. Remove gh CLI install from Containerfile. Add tests/test_git.py with 14 unit tests covering all changed functions. Signed-off-by: Eike Waldt <waldt@b1-systems.de> On-behalf-of: SAP <eike.waldt@sap.com> Assisted-by: Kilo:claude-opus-4-8 Assisted-by: Kilo:claude-sonnet-4-6

gardenlinux-docs-bot · 2026-07-22T09:06:01Z

📚 Documentation Preview PR: gardenlinux/docs#132
😎 Deploy Preview: https://deploy-preview-132--gardenlinux-docs.netlify.app

A PR has been created/updated to preview the documentation changes from this PR.

yeoldegrove self-assigned this Sep 9, 2025

yeoldegrove force-pushed the feat/bump_python_gardenlinux_lib branch 2 times, most recently from e5121c4 to f911488 Compare September 9, 2025 13:42

yeoldegrove force-pushed the feat/bump_python_gardenlinux_lib branch from f911488 to 28a7960 Compare September 23, 2025 11:37

yeoldegrove linked an issue Sep 23, 2025 that may be closed by this pull request

refactoring of GLRD codebase #4

Open

yeoldegrove force-pushed the feat/bump_python_gardenlinux_lib branch from 28a7960 to 16cd1c4 Compare October 8, 2025 10:35

yeoldegrove changed the title ~~refactor to work with gardenlinux 0.9.1~~ refactor to work with gardenlinux 0.10.0 Oct 8, 2025

yeoldegrove force-pushed the feat/bump_python_gardenlinux_lib branch from 16cd1c4 to 4e3988d Compare October 8, 2025 11:27

yeoldegrove force-pushed the feat/bump_python_gardenlinux_lib branch from 4e3988d to 7e02f13 Compare April 9, 2026 10:14

yeoldegrove changed the title ~~refactor to work with gardenlinux 0.10.0~~ refactor to work with gardenlinux 0.10.20 Apr 9, 2026

yeoldegrove added 13 commits July 21, 2026 13:22

refactor to work with gardenlinux 0.10.20

649e403

Fix lint errors and format code with black

5ddf879

- Remove unused import in schema.py - Fix line length issues in schema.py - Add proper blank line after function in util.py - Add proper type hints for parse_isodatetime - Format all modified files with black

refactor: Fix lint errors in models.py and git.py

83ce456

- Remove unused datetime import from models.py - Remove unused major_version variable in git.py - Fix trailing whitespace in git.py - Format code with black All tests passing.

yeoldegrove force-pushed the feat/bump_python_gardenlinux_lib branch 2 times, most recently from d0fe837 to 2eecb72 Compare July 22, 2026 05:06

yeoldegrove changed the title ~~refactor to work with gardenlinux 0.10.20~~ refactor to work with gardenlinux 0.10.21 Jul 22, 2026

yeoldegrove force-pushed the feat/bump_python_gardenlinux_lib branch from 55852a7 to 8bf6ea1 Compare July 22, 2026 08:45

yeoldegrove added 2 commits July 22, 2026 10:51

yeoldegrove force-pushed the feat/bump_python_gardenlinux_lib branch from 8bf6ea1 to ec58af6 Compare July 22, 2026 08:52

gardenlinux-docs-bot Bot mentioned this pull request Jul 22, 2026

docs(glrd): Update commit lock from PR #13 gardenlinux/docs#132

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor to work with gardenlinux 0.10.21#13

refactor to work with gardenlinux 0.10.21#13
yeoldegrove wants to merge 15 commits into
mainfrom
feat/bump_python_gardenlinux_lib

yeoldegrove commented Sep 9, 2025 •

edited

Loading

Uh oh!

gardenlinux-docs-bot Bot commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

yeoldegrove commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code refactoring

glrd/git.py — replace gh/git CLI with python-gardenlinux-lib

release.py (renamed from models.py) — single source of truth

manage.py — remove 547 lines of duplicates

schema.py — unify schema_v1.py and schema_v2.py

Other refactors

Tests

Documentation

Uh oh!

gardenlinux-docs-bot Bot commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

yeoldegrove commented Sep 9, 2025 •

edited

Loading