[image-spec]: nix runner SSH build pattern matching exa-deploy#44
Open
jld-adriano wants to merge 2 commits intomasterfrom
Open
[image-spec]: nix runner SSH build pattern matching exa-deploy#44jld-adriano wants to merge 2 commits intomasterfrom
jld-adriano wants to merge 2 commits intomasterfrom
Conversation
Build+push on native runners via SSH instead of local nix run with remote builders. Each arch builds natively on its own runner: - _nix_build_on_runner: nix build --store ssh-ng://root@runner - _nix_run_on_runner: SSH to runner, execute copyTo - Same runner pool and SSH key (nix-runner-ssh-key) as exa-deploy - Removes local cross-build complexity (buildSystem, local_system detection) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tracking issue
Related to exa-labs/monorepo#18302
Why are the changes needed?
The previous nix build approach ran builds locally and used nix remote builders for cross-architecture builds, requiring a
buildSystemparameter and complex cross-docker package generation in the flake. This differed from how exa-deploy builds images (SSH to dedicated nix runners, build+push on-runner). Aligning the two systems simplifies the flake, removes cross-build complexity, and ensures each architecture builds natively on its own runner.What changes were proposed in this pull request?
Replaces the local
nix run ... #copyToapproach with an SSH-to-runner pattern matchingexa-deploy'smkNixBuild/mkNixRun:_nix_build_on_runner— runsnix build --store ssh-ng://root@{runner}to evaluate locally and build on the remote runner's store (same as exa-deploy'smkNixBuild)_nix_run_on_runner— SSHs to the runner to execute the builtcopy-tobinary for ECR push (same as exa-deploy'smkNixRun)_load_nix_runner_ssh_key— fetchesnix-runner-ssh-keyfrom AWS Secrets Manager (same as exa-deploy'sloadSshKey)Same runner pool and SSH key as exa-deploy. Removes local system detection (
platform.system()/platform.machine()) and cross-build branching since each arch now builds natively on its own runner.Human review checklist
flyte_aws.pyon hephaestus after the companion monorepo PR lands. The previous cross-build approach was validated but this new runner-SSH path has not been.image_spec.image_name()and ECR token flow into the SSH remote command string via f-string interpolation, notshlex.quote(). ECR tokens are base64-safe, image names are registry URLs, but worth eyeballing for injection surface._load_nix_runner_ssh_key()is called even forpush=False(build-only). Local-only nix builds now require AWS credentials.exa-deploy/src/lib.ts, always picksrunners[0]with no fallback or health check.Updates since last revision
0o600permissions viaos.open()(no race window)_nix_run_on_runnernow logs the redacted SSH command usingre.subfor targeted credential maskingHow was this patch tested?
Not yet tested end-to-end. The previous cross-build approach (which this replaces) was validated with
flyte_aws.pyon the hephaestus cluster. This new runner-based approach needs the same validation after the companion monorepo PR removes thebuildSystem/cross-docker flake complexity.Check all the applicable boxes
Related PRs
buildSystem/crossDockerPackagesfrom the flakeLink to Devin run: https://app.devin.ai/sessions/a0791e463dec4cfcacc33e89aa5c1d9f
Requested by: @jld-adriano