feat: prefer specific OS, gate .git on unpacker, probe zst/7z, deploy script - #1074
Merged
Conversation
`.git` was pushed unconditionally into getSortedFormats's candidate
ext list, while sibling unpacker formats (.tar.xz, .tar.zst, .zip,
.7z) are gated on whether the caller's `formats` argument signals
the client has the corresponding tool.
Make `.git` consistent: only add it to the candidate list when
formats includes 'git'. The default WEBI_FORMATS ('tar,exe,zip,xz,
dmg') doesn't include git, so the change is a no-op for the
current default. Clients that want git-source packages installed
can pass `?formats=tar,exe,zip,xz,dmg,git` (or set the equivalent
in a future client-side probe).
For packages that have only a git-source asset (e.g. some vim
plugins), the existing fallback to `packages[0]` still returns the
git entry — behavior unchanged. The only observable change is for
packages where both a binary and a git fallback exist for the same
triplet: previously the git entry could win over the binary; now
it wins only when the client opts in.
webi/webi.sh: detect unzstd/zstd alongside the existing git/unxz/ unzip/tar probes. Sends `?formats=...,zst` when zstd is available so the server can pick a .tar.zst build only on hosts that can extract it. webi/webi-pwsh.ps1: replace the hardcoded `formats=zip,exe,tar,git` TODO with real Get-Command probing for git, zstd, and 7z.
coolaj86
force-pushed
the
fix-builds-cacher-git-gate
branch
2 times, most recently
from
May 7, 2026 06:17
4b2deb0 to
0ec38d4
Compare
In _enumerateTriplets, the order of `oses` and `arches` was ANYOS/ANYARCH first, specific second. This caused findMatchingPackages to pick the most-generic triplet (e.g. ANYOS-ANYARCH-none) before trying specific OS triplets — and packages that have a wildcard git fallback alongside per-platform binaries would resolve to the git source instead of the binary, even when the client never asked for git as an unpacker. Reverse the order so specific platforms win: - oses: hostTarget.os, posix_2017, posix_2024, ANYOS - arches: arches.concat(['ANYARCH']) Concrete example: serviceman has both posix_2017/*/tar.gz and */*/git in the cache. Pre-fix, findMatchingPackages picks ANYOS-ANYARCH-none (containing only the .git entry). The .git gate in getSortedFormats then correctly excludes git from format candidates, but the chosen triplet has nothing else, so selectPackage falls through to packages[0] = git entry. Post-fix, findMatchingPackages picks posix_2017-ANYARCH-none first (containing [tar.gz, zip]), and selectPackage returns tar.gz.
Pulls in webinstall/webi-build-classifier#21 (merged 2026-05-07, SHA 574eff5) and the host-target x64/win32 fix from #20 (SHA 71c0768) that landed alongside it. #21 fixes `maybeInstallable` rejecting any package version ending in `.1` whose download URL is a GitHub source-archive endpoint (`/tarball/vX.Y.1` or `/zipball/vX.Y.1`). Without it, this PR's `_enumerateTriplets` priority fix is undermined: even after picking the correct posix_2017 triplet, the newest version (e.g. serviceman v1.0.1) is silently dropped by the classifier and the resolver falls back to v1.0.0. Confirmed on next.webi.sh after deploying this branch with the bumped submodule: `serviceman@stable.sh` now resolves to v1.0.1/zip on macOS arm64 (was v1.0.0/zip with the pre-rebase pre-fix submodule).
coolaj86
force-pushed
the
fix-builds-cacher-git-gate
branch
from
May 7, 2026 06:22
0ec38d4 to
da10371
Compare
coolaj86
added a commit
that referenced
this pull request
May 8, 2026
Trim the explanatory paragraphs in _classify and findMatchingPackages to one or two lines stating the WHY (the gotcha or the invariant); drop time-bound references (#1074), illustrative enumerations, and restated background that the surrounding code already conveys.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five related changes that together make the resolver pick the right binary instead of accidentally installing source archives, git clones, or just dropping versions on the floor:
_webi/builds-cacher.js—_enumerateTriplets: prefer specific OS/arch overANYOS/ANYARCH._webi/builds-cacher.js—getSortedFormats: gate.giton the caller'sformatslist, mirroring.tar.xz,.tar.zst,.zip,.7z.webi/webi.sh: probeunzstd/zstdalongside the existing git/unxz/unzip/tar probes.webi/webi-pwsh.ps1: replace the hardcodedformats=zip,exe,tar,gitTODO withGet-Commandprobes for git, zstd, 7z._scripts/deploy-webid: rsync deploy with submodule precondition,_cachesymlink restore, anchored worker pkill, smoke tests._webi/build-classifier: bump submodule to pull in webi-build-classifier#21 — fixesmaybeInstallablerejecting any package version ending in.1.Commit 6 bumps the submodule to a SHA on
webinstall/webi-build-classifier#21's feature branch. That PR should land first so the bumped SHA is reachable fromwebi-build-classifier'smain. Without #21,serviceman v1.0.1(and any other patch.1 version of a source-archive-only package) is still silently dropped by the classifier and the user getsv1.0.0instead.Motivation
serviceman(and any package with a*/*/gitfallback alongside per-OS binaries) was resolving togitinstead of the binary, even on hosts withgitabsent:Two distinct bugs:
A. Triplet ordering —
_enumerateTripletsproducedANYOS-*triplets before specific OS triplets:findMatchingPackagesreturned the git-only triplet first. The.git-as-unpacker gate (commit 2) is correct but downstream — by the timeselectPackageis called the wrong triplet has already been picked.B. Classifier silent drop —
Triplet.maybeInstallablecheckedbuild.download.endsWith(ext)againstTERMS_EXTS_NON_BUILDwhich contains.1(intended formanpage.1). GitHub source-archive URLs of the form.../tarball/v1.0.1end in.1, so the classifier silently rejected the entire v1.0.1 release. Fixed in webi-build-classifier#21 by checkingbuild.nameinstead.Behavior
After this PR + the build-classifier PR:
servicemanmacos/linux any host (default formats):v1.0.1/zip(wasgit→ would fail without git).servicemanWindows: still no Windows binary in the cache (Go-cache content gap, not addressed by this PR — needstherootcompany/servicemanfetched alongsidebnnanet/serviceman).vim-commentary(only-git package): unchanged.selectPackage'spackages.length === 1early return covers single-option triplets.webi <pkg>on a host with zstd: now sends?formats=...,zst.webi-pwsh.ps1on Windows: probes git/zstd/7z viaGet-Command.Test plan
npm testpasses.beta.webi.shandnext.webi.sh. Verified/api/installers/serviceman@stable.sh:v1.0.1/zip(wasv1.0.1/gitpre-PR; would bev1.0.0/zipwithout commit 6).v1.0.1/zip).v0.9.8/git(still broken — cache gap, separately tracked).vim-commentarystill resolves to git correctly.Notes
_scripts/deploy-webidincludes a submodule precondition that catches "I forgot togit submodule update --init" before the rsync silently pushes an empty_webi/build-classifier/. That scenario broke deploys repeatedly during this PR's testing.Commits
Depends on: webinstall/webi-build-classifier#21