ci: Build doc versions in parallel and fix the failing app builds#22
Merged
Conversation
The docs build ran every version sequentially in a single job. The versions are independent of each other, so they are now built by a matrix, with one job per version, and a single job that assembles the results and pushes. The push stays in one job so the builders never race on the branch. publish.sh gains `list`, `build` and `assemble` subcommands, which the workflow drives. Running it without arguments still performs the full sequential build locally, so the build logic is not duplicated between the script and the workflow. Also updates the actions and their runtimes: checkout v4 -> v7, setup-node v3 -> v7, Node 16 (EOL since 2023) -> 22, and Melos 7.0.0-dev.7 -> 8.2.2. The old Melos pin could not satisfy v1.38.0, which requires ^8.1.0.
The Flutter apps embedded in the docs failed to compile for every version older than v1.36.0, with "Couldn't resolve the package 'rive_native'". The versions are built one after another in the same clone, and `git checkout -f` leaves ignored files alone. `.dart_tool/` holds a generated web plugin registrant, so once a version using rive ^0.14 has been built, the registrant importing `package:rive_native` survives into the next version. Versions using rive ^0.13 do not depend on rive_native, so the stale import no longer resolves and the app fails to compile. Reproduced with a clone checked out at v1.36.0, built, then moved to v1.35.1: the build fails exactly as in CI, and succeeds again once `git clean -xfd` is added after the checkout.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Two things, both found while investigating why run 29694895994 took 4h17m: the embedded
Flutter apps were failing to compile for every version older than v1.36.0, and the versions
were built sequentially when they are independent of each other.
Fix the failing app builds
Every version older than v1.36.0 failed to compile its embedded Flutter apps with:
590 of the 647 compiles in that run failed this way. It is not a dependency problem: rive
0.13.20does not depend onrive_nativeat all, only0.14.0+ does. It is state leakingbetween versions.
All versions are built in the same clone, newest first.
git checkout -fdoes not touchignored files, and
.dart_tool/,.flutter-plugins-dependenciesand**/build/are allgitignored.
.dart_tool/holds a generated web plugin registrant, so once v1.36.0 (rive^0.14) has been built, a registrant importingpackage:rive_nativesurvives the checkoutinto v1.35.1 (rive
^0.13), which does not depend on it. The import no longer resolves andevery app build fails from there on down. The version boundary matches exactly where flame
moved from rive
^0.13.20to^0.14.0.Adding
git clean -xfdafter the checkout makes each version start from a pristine tree.Note
-xis required: plaingit clean -fdwould skip these files precisely because theyare ignored.
Reproduced in one clone, which is exactly what CI does:
✓ Built build/webgit checkout -fonlyCouldn't resolve the package 'rive_native'git clean -xfdfirst✓ Built build/webThis is a correctness fix as much as a speed one: the embedded apps in every published
version below v1.36.0 are currently broken.
Build the versions in parallel
The versions are independent, so they are now built by a matrix with one job per version, plus
a single job that assembles the results and pushes.
publish.shgainslist,buildandassemblesubcommands which the workflow drives:listclones the Flame repo blob-filtered and no-checkout (it only needs the refs) andemits
latestandtargetsfor the matrix.build <version> <latest>builds one version intoout/.assemble <targets>merges the artifacts intodocs/and pushes.Running
./publish.shwithout arguments still performs the full sequential build locally, sothe build logic is not duplicated between the script and the workflow.
The push stays in the single
publishjob so the builders never race on the branch, andfail-fast: falsemeans one broken version does not prevent the others from being published.Note the matrix also makes the state leak structurally impossible, since every version gets a
fresh runner and a fresh clone. The
git clean -xfdis kept anyway: it is what makes thelocal sequential path correct, and it should not depend on how CI happens to be sharded.
Expected wall clock is roughly 5 minutes, down from 4h17m.
Action and runtime updates
Bundled here since they touch the same file:
actions/checkoutactions/setup-nodesubosito/flutter-action@v2andbluefireteam/melos-action@v3are already on their latestmajors. The old Melos pin could not satisfy v1.38.0, which requires
^8.1.0, somainandthe newest tag were relying on
melos bootstrap || echo ...swallowing the failure. Melos8.2.2 was checked against v1.25.0, the oldest tag in the build list: it does not reject a
workspace declaring
^7, and loads all packages and the script registry, somelos run doc-setupresolves.Related
flame-engine/flame#3953 stops the
flutter-appdirective from recompiling an app thatalready failed, so a broken app costs one compile rather than one per referencing page. It is
independent of this PR, but it is what bounds the damage if a version ever does fail again.
Testing instructions
The workflow is
workflow_dispatchonly, so the real test is dispatching it from this branchand checking that:
versionsjob outputs 17 targets withlatest=v1.38.0.buildjobs run in parallel, each uploading adocs-<version>artifact.Couldn't resolve the package 'rive_native', and each version logs 5successful
Compiling Flutter applines rather than dozens of repeats.publishjob produces adocs/tree matching the sequential build: one directory perversion, a
docs/latestcopy, anddocs/versions.txtlistingmainfirst then newest tooldest.
docs/latesthas nonoindexmarker while every versioned directory does. The ordering ingenerate_docs_for_versionmatters here:latestis copied before the marker is added.Worth spot-checking a page like
flame/effects/anchor_effectson an older version oncepublished, since those embedded apps should now actually run.