ci(build): native amd64+arm64 runners + merge-job pattern#27
Draft
CybotTM wants to merge 1 commit into
Draft
Conversation
Splits the multi-arch build from one QEMU-emulated job per
(track, composer) into two native-arch parallel jobs that build by
digest, plus a merge job per cell that assembles the final manifest
list, attests, and signs it.
Before: ubuntu-latest with `setup-qemu-action` and
`platforms: linux/amd64,linux/arm64` → arm64 emulated → ~40 min wall
per cell. With 5 active matrix cells (3 tracks × 2 composer modes
minus tag/rolling exclude), the full daily build took ~40 min.
After: each (track, composer, platform) builds on its native runner —
amd64 on `ubuntu-latest`, arm64 on `ubuntu-24.04-arm` — both ~12 min
in parallel. Then a per-(track, composer) merge job joins the two
digests with `docker buildx imagetools create` and pushes the
manifest list under the canonical tags. Daily build now ~15 min wall.
Implementation:
- Build job (now per-arch):
- `runs-on: ${{ matrix.platform.runner }}` — `ubuntu-latest` / `ubuntu-24.04-arm`
- QEMU step removed (native)
- `docker/build-push-action` outputs `type=image,push-by-digest=true,
name-canonical=true` (no tags pushed yet)
- Per-arch digest exported and uploaded as a workflow artifact
- Merge job (new):
- `needs: build`, runs after both per-arch builds complete
- Per (track, composer) matrix matching the build matrix
- Downloads the per-arch digest artifacts
- Re-resolves the ref and re-computes the tag list (same logic as
the build job)
- `docker buildx imagetools create -t <tag1> -t <tag2> ...
<image>@sha256:<amd64> <image>@sha256:<arm64>`
- Captures the manifest-list digest, then runs
`attest-build-provenance` and `cosign sign` on THAT digest
- Sanity check: aborts if fewer than 2 per-arch digests arrived
(catches silent build-job arch failures that would otherwise
publish a degenerate single-arch manifest)
What stays the same:
- The `tag/rolling` matrix exclude (symfony/dom-crawler audit advisory)
- `continue-on-error` for the rolling variants
- The PR-event gate that runs only the `tag` track for PR builds
(PR builds now produce 2 parallel per-arch builds — both push-by-
digest=false, so no registry writes happen on PR)
- All attestation + cosign outputs land under the manifest-list digest,
identical observable behaviour for consumers
Signed-off-by: Sebastian Mendel <info@sebastianmendel.de>
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — needs a CI run before non-draft; touches the security-sensitive attestation + cosign signing chain.
What
Splits the multi-arch build from one QEMU-emulated job per (track, composer) into two native-arch parallel jobs that build by digest, plus a merge job per cell that assembles the manifest list, attests, and signs it.
Before / After
mergejobImplementation
Build job (now per-arch)
runs-on: ${{ matrix.platform.runner }}—ubuntu-latest/ubuntu-24.04-armdocker/build-push-actionoutputstype=image,push-by-digest=true,name-canonical=true— no tags pushed yetdigests-<track>-<composer>-<arch>Merge job (new)
needs: build, per(track, composer)matrix matching the build matrixpattern: digests-<track>-<composer>-*)docker buildx imagetools create -t <tag1> -t <tag2> ... <image>@sha256:<amd64-digest> <image>@sha256:<arm64-digest>docker buildx imagetools inspect <first-tag> --format '{{json .Manifest}}' | jq -r '.digest'actions/attest-build-provenanceandcosign signon the manifest-list digestSanity guard
Merge job aborts if fewer than 2 per-arch digests arrived (catches silent per-arch build failures that would otherwise publish a degenerate single-arch manifest under the multi-arch tag).
What stays identical for consumers
8.5.0,8.5.0-rolling,8.5.0-YYYYMMDD,8.5,8,latest,rolling,sha-pinned-<sha>,sha-rolling-<sha>, branch tags) — same names, same multi-arch manifest behind themgh attestation verify,cosign verify)tag/rollingmatrix exclude (symfony/dom-crawler audit advisory)continue-on-error: rollingsemanticstag-track-only)Verification
Once CI runs on this PR:
tag/pinned(PR events skip non-tag tracks)push: falseon PR;push-by-digest=true,push=trueon main/schedule)if: github.event_name != 'pull_request')After merge:
Why draft
subject-digestnow points at the manifest-list digest instead of the per-platform digest. The new behaviour matches docker's recommended multi-platform pattern (https://docs.docker.com/build/ci/github-actions/multi-platform/) but it's a real change in the verification surface for consumersarm64matrix cell on PR events doubles the PR-event runner footprint; if that becomes a cost concern we can gate arm64 to non-PR events in the matrix-excludeResolves the "arm64 native runners" roadmap item that was in README before PR #22 removed it as in-progress work.