forked from ethereum-optimism/optimism
-
Notifications
You must be signed in to change notification settings - Fork 0
feat(op-reth): add StreamingFast Docker image build + release workflow #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
maoueh
merged 4 commits into
feature/update-to-latest-op-reth
from
feature/docker-image-building
Jun 11, 2026
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
051d01d
feat(op-reth): add StreamingFast Docker image build + release workflow
maoueh a5bfbc3
fix(ci): gate release step to tag refs
Copilot 0e77483
revert(ci): remove redundant release step tag gate
Copilot cd43dc6
ci(sf-release): branch/PR triggers, ubuntu-24.04, alpha + slash-safe …
maoueh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
| # StreamingFast Docker image build, push and (on tag) release. | ||
| # | ||
| # Maintained by StreamingFast and intentionally kept separate from upstream CI | ||
| # so that upstream merges stay clean. | ||
| # | ||
| # Triggers: `firehose/*` branch pushes, `*-fh*` tags, pull requests and manual | ||
| # dispatch. Pull requests opened from a fork get a read-only GITHUB_TOKEN, so the | ||
| # login/push steps are skipped for them (the image still builds, to validate the | ||
| # Dockerfile); same-repo ("inner") PRs and pushes authenticate and push normally. | ||
| # | ||
| # Single architecture (linux/amd64): the op-reth `maxperf` full-node build is too | ||
| # heavy to reliably build on GitHub-hosted arm64 runners, so we do not produce a | ||
| # multi-arch manifest. Re-introduce a matrix + manifest-merge job here if arm64 | ||
| # becomes worthwhile. | ||
| name: Build, push and release (if tag) | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - "firehose/*" | ||
| tags: | ||
| - "*-fh*" | ||
| pull_request: | ||
| workflow_dispatch: | ||
|
|
||
| env: | ||
| REGISTRY: ghcr.io | ||
| IMAGE_NAME: ${{ github.repository }} | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-24.04 | ||
|
|
||
| permissions: | ||
| contents: read | ||
| packages: write | ||
|
|
||
| env: | ||
| # Pushes (branch/tag), same-repo PRs and manual dispatch can authenticate | ||
| # to ghcr and push images. A PR opened from a fork gets a read-only | ||
| # GITHUB_TOKEN with no `packages: write`, so we must skip login and push for | ||
| # it — the build still runs to validate the Dockerfile compiles. The value | ||
| # is the string "true"/"false". | ||
| IS_EXTERNAL_PR: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository }} | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log in to the Container registry | ||
| if: env.IS_EXTERNAL_PR != 'true' | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Extract version | ||
| id: extract-version | ||
| run: | | ||
| version="edge" | ||
| if [[ "${GITHUB_REF}" == refs/tags/* ]]; then | ||
| version=${GITHUB_REF#refs/tags/} | ||
| fi | ||
| # Docker tags forbid `/`; a tag (or ref) containing slashes would make | ||
| # the `<sha>-<version>` tag invalid. Match docker/metadata-action's own | ||
| # sanitization by replacing `/` with `-`. | ||
| version=${version//\//-} | ||
| echo "VERSION=${version}" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Generate docker tags/labels from github build context | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} | ||
| # The version-suffixed sha tag uniquely identifies the image for a given | ||
| # build, appending the extracted version (either `edge` or the tag). This | ||
| # avoids a race condition when a tag and its branch are pushed together: | ||
| # both builds run on the same commit and would otherwise push the same | ||
| # `<sha>` tag, with the last writer winning and possibly clobbering the | ||
| # tag build's version label. The `<sha>-<version>` tag stays distinct | ||
| # (`<sha>-edge` vs `<sha>-v1.2.3-fh3.0`), and the `release` job below | ||
| # reads `<sha>-<tag>` so it always picks up the correct image. | ||
| tags: | | ||
| type=ref,event=tag,prefix= | ||
| type=ref,event=branch | ||
| type=ref,event=pr | ||
| type=sha,prefix= | ||
| type=sha,prefix=,suffix=-${{ steps.extract-version.outputs.VERSION }} | ||
| type=edge | ||
| flavor: | | ||
| latest=${{ startsWith(github.ref, 'refs/tags/') && !contains(github.ref, 'beta') && !contains(github.ref, 'rc') && !contains(github.ref, 'alpha') }} | ||
|
|
||
| - name: Build and push Docker image (linux/amd64) | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: rust | ||
| file: ./rust/op-reth/Dockerfile.sf | ||
| platforms: linux/amd64 | ||
| push: ${{ env.IS_EXTERNAL_PR != 'true' }} | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
| provenance: false | ||
| build-args: | | ||
| VERSION=${{ steps.extract-version.outputs.VERSION }} | ||
|
|
||
| release: | ||
| if: startsWith(github.ref, 'refs/tags/') | ||
| needs: build | ||
| runs-on: ubuntu-24.04 | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| steps: | ||
| - name: Docker login | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Extract image | ||
| id: image | ||
| run: | | ||
| version=${GITHUB_REF#refs/tags/} | ||
| # Match the sanitization done in the build job's "Extract version" step | ||
| # so this ID resolves to the same `<sha>-<version>` image tag. | ||
| version=${version//\//-} | ||
| echo "ID=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${GITHUB_SHA::7}-${version}" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Extract assets | ||
| run: | | ||
| # The --platform is not strictly needed for a single-arch image, but it | ||
| # silences the buildx warning. | ||
| docker cp $(docker create --platform=linux/amd64 ${{ steps.image.outputs.ID }}):/usr/local/bin/op-reth ./op-reth_linux_amd64 | ||
|
|
||
| - name: Extract Changelog | ||
| id: changelog | ||
| run: | | ||
| curl -L https://github.com/streamingfast/sfreleaser/releases/download/v0.12.1/sfreleaser_linux_x86_64.tar.gz | tar -xz | ||
| chmod +x sfreleaser | ||
|
|
||
| ./sfreleaser changelog extract-section \ | ||
| github://token:${{ github.token }}@${{ github.repository }}/$GITHUB_SHA/CHANGELOG.sf.md \ | ||
| --github-output="changelog:$GITHUB_OUTPUT" | ||
|
|
||
| - name: Release | ||
| uses: softprops/action-gh-release@v2 | ||
| with: | ||
| body: ${{ steps.changelog.outputs.changelog }} | ||
| prerelease: ${{ contains(github.ref, 'beta') || contains(github.ref, 'rc') || contains(github.ref, 'alpha') }} | ||
| files: | | ||
| ./op-reth_linux_amd64 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # StreamingFast op-reth Changelog | ||
|
|
||
| All notable StreamingFast-specific changes to this fork are documented in this | ||
| file. It tracks only the `.sf`-suffixed StreamingFast additions (Firehose | ||
| instrumentation, Docker image, release flow) on top of upstream op-reth. | ||
|
|
||
| The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). | ||
| The `sf-release.yml` workflow publishes the top-most version section here as the | ||
| GitHub release notes (via `sfreleaser changelog extract-section`), so keep the | ||
| most recent release at the top. | ||
|
|
||
| ## Unreleased | ||
|
|
||
| ### Added | ||
|
|
||
| - StreamingFast Docker image build, push and release flow (`Dockerfile.sf`, | ||
| `.github/workflows/sf-release.yml`). Pushing a `*-fh*` tag builds the | ||
| Firehose-instrumented `op-reth` binary, publishes a `linux/amd64` image to | ||
| `ghcr.io`, and creates a GitHub release with the binary attached and these | ||
| notes as the body. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| # StreamingFast variant of op-reth/DockerfileOp. | ||
| # | ||
| # This file is maintained by StreamingFast and is intentionally kept separate | ||
| # from upstream's `op-reth/DockerfileOp` so that upstream merges stay clean. | ||
| # It builds the Firehose-instrumented `op-reth` binary and installs it at a | ||
| # stable path (`/usr/local/bin/op-reth`) that the `sf-release.yml` workflow can | ||
| # `docker cp` out of the image to attach as a GitHub release asset. | ||
| # | ||
| # Keep this in sync with `op-reth/DockerfileOp` to avoid drift: when the upstream | ||
| # Dockerfile changes (base image, build profile, build command), mirror the | ||
| # relevant change here. | ||
| # | ||
| # Build context is the `rust/` workspace directory (same as the upstream | ||
| # `op-reth` docker-bake target), so crate paths are rooted at `/app/op-reth`. | ||
|
|
||
| FROM lukemathwalker/cargo-chef:latest-rust-1.94 AS chef | ||
| WORKDIR /app | ||
|
|
||
| LABEL org.opencontainers.image.source=https://github.com/streamingfast/op-reth | ||
| LABEL org.opencontainers.image.licenses="MIT OR Apache-2.0" | ||
|
|
||
| RUN apt-get update && apt-get -y upgrade && apt-get install -y libclang-dev pkg-config | ||
| RUN cargo install cargo-auditable | ||
|
|
||
| # Builds a cargo-chef plan | ||
| FROM chef AS planner | ||
| COPY . . | ||
| RUN cargo chef prepare --recipe-path recipe.json | ||
|
|
||
| FROM chef AS builder | ||
| COPY --from=planner /app/recipe.json recipe.json | ||
|
|
||
| ARG BUILD_PROFILE=maxperf | ||
| ENV BUILD_PROFILE=$BUILD_PROFILE | ||
|
|
||
| ARG RUSTFLAGS="" | ||
| ENV RUSTFLAGS="$RUSTFLAGS" | ||
|
|
||
| ARG FEATURES="" | ||
| ENV FEATURES=$FEATURES | ||
|
|
||
| # Version of the release being built (e.g. the pushed tag). Stamped as an OCI | ||
| # image label below. The reth binary derives its own `--version` string from the | ||
| # upstream git metadata at compile time, so this is informational only. | ||
| ARG VERSION="dev" | ||
|
|
||
| # Allow cook to fail on patched crates (op-alloy-network) whose crates.io | ||
| # version conflicts with alloy 1.8. All other dependencies are still cached. | ||
| # The real build below uses the workspace [patch.crates-io] sources. | ||
| RUN cargo chef cook --profile $BUILD_PROFILE --features "$FEATURES" --recipe-path recipe.json --manifest-path /app/op-reth/bin/Cargo.toml || true | ||
|
|
||
| COPY . . | ||
| RUN cargo auditable build --profile $BUILD_PROFILE --features "$FEATURES" --bin op-reth --manifest-path /app/op-reth/bin/Cargo.toml | ||
|
|
||
| RUN ls -la /app/target/$BUILD_PROFILE/op-reth | ||
| RUN cp /app/target/$BUILD_PROFILE/op-reth /app/op-reth | ||
|
|
||
| FROM chainguard/wolfi-base:latest AS runtime | ||
|
|
||
| ARG VERSION="dev" | ||
| LABEL org.opencontainers.image.version="${VERSION}" | ||
|
|
||
| RUN apk add --no-cache ca-certificates openssl libstdc++ strace | ||
|
|
||
| WORKDIR /app | ||
| COPY --from=builder /app/op-reth /usr/local/bin/ | ||
| RUN chmod +x /usr/local/bin/op-reth | ||
| COPY op-reth/LICENSE-* ./ | ||
|
|
||
| EXPOSE 30303 30303/udp 9001 8545 8546 7545 8551 | ||
| ENTRYPOINT ["/usr/local/bin/op-reth"] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add
if: github.ref_type == 'tag'hereref: https://github.com/softprops/action-gh-release#-limit-releases-to-pushes-to-tags
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot Apply suggestion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot Ignore, incorrect.
The full job is actually already gated, no need to gate again
if: startsWith(github.ref, 'refs/tags/')see line 86There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Applied in a5bfbc3: added
if: github.ref_type == 'tag'on theReleasestep in.github/workflows/sf-release.yml.