Add NXF_DEV launcher support for automated dev builds#7219
Open
ewels wants to merge 3 commits into
Open
Conversation
Adds an NXF_DEV environment variable to the launcher to run automated development builds published in the nextflow-io/nextflow-dev-builds repository: NXF_DEV=master for the latest master build, NXF_DEV=pr-1234 for the latest build of a pull request. Also adds a workflow that posts a comment on each pull request with ready-made commands for testing that PR's dev build, updated in place on each push. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Phil Ewels <phil.ewels@seqera.io>
✅ Deploy Preview for nextflow-docs-staging ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
When the DEV_BUILDS_DISPATCH_TOKEN secret is configured, PR pushes and master pushes dispatch an immediate build in nextflow-dev-builds instead of waiting for the scheduled run there. Without the secret, or if the dispatch fails, everything falls back to the scheduled build and the PR comment says which mode applied. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Phil Ewels <phil.ewels@seqera.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Testing an unreleased Nextflow change today means waiting for an edge release, or building from source and hosting the result somewhere. That makes it slow to verify a fix on master, hard for issue reporters to confirm that a fix works for them, and effectively impossible for most users to try out functionality from an open PR.
With this change, anyone can do:
No build tools, no waiting for a release. Set inline as above, nothing persists in the environment, and the next plain
nextflowinvocation behaves exactly as before.How the system works
The builds come from nextflow-io/nextflow-dev-builds, which is already live:
masterand recently active PRs within ~30 minutes of a push, publishing each build as a pre-release (fat jar, standalone executable, checksums). PR code is compiled in a job with a read-only token; publishing happens in a separate job that never executes source code.latest/master,latest/pr-1234) track the newest build per channel. Old builds are pruned automatically, and builds for closed PRs are deleted.NXF_BASEplusNXF_VER, since the launcher's download URL layout is matched exactly.What this PR adds
NXF_DEVin the launcher: resolves the channel's latest build version from the pointer file and pointsNXF_BASEat the dev-builds repo. Zero overhead when unset (one bash conditional).self-updaterefuses to run whileNXF_DEVis set, so a user cannot accidentally replace their installed launcher with a dev build.dev-build-comment.ymlworkflow: posts a comment on each PR with copy-paste commands for testing that PR's build, updated in place on each push (never repeated). It usespull_request_targetso it also works for fork PRs; this is safe because it never checks out or executes PR code, it only reads PR metadata and theVERSIONfile, whose content is validated against a character allowlist before use.dev-build-dispatch.yml) dispatch a build in the dev-builds repo right away, instead of waiting for the scheduled run. This needs theDEV_BUILDS_DISPATCH_TOKENsecret (see follow-up below). Without it, or if the dispatch fails, everything falls back gracefully to the scheduled ~30 minute build, and the PR comment states which mode applied so a misconfiguration is visible without blocking anyone.NXF_DEVentry in the environment variable reference.Follow-up after merge
To enable immediate builds (a few minutes instead of up to 30), add a repository secret. Until then the workflows fall back to the scheduled builds automatically.
Setting up
DEV_BUILDS_DISPATCH_TOKENnextflow-ionextflow-io/nextflow-dev-buildsDEV_BUILDS_DISPATCH_TOKENBuild dev releasesrun appears in nextflow-dev-builds actions.Notes
NXF_BASE=https://github.com/nextflow-io/nextflow-dev-builds/releases/download NXF_VER=<build version> nextflow run <pipeline>. The PR comment shows both forms.Companion docs PR: nextflow-io/nextflow-dev-builds#1
🤖 Generated with Claude Code