test: free RAM between multi-package describes to fix CI OOM - #161
Closed
ryukzak wants to merge 12 commits into
Closed
test: free RAM between multi-package describes to fix CI OOM#161ryukzak wants to merge 12 commits into
ryukzak wants to merge 12 commits into
Conversation
Top-level `await` inside `describe` built every result during discovery and pinned it via closure for the file's lifetime, causing OOM during sql-on-fhir.test.ts on ubuntu-latest. Move generation into `beforeAll` and null out the result in `afterAll` (+ Bun.gc) so each section's ~tens-of-MB of in-memory generated files can be reclaimed between sections.
The previous commit moved generation from describe-discovery (no timeout) into `beforeAll`, which inherits bunfig's `timeout = 30000`. SQL-on-FHIR fetches `https://build.fhir.org/.../package.tgz` on cold cache from GH runners and routinely exceeds 30s; the hook timed out, got retried, and OOM-killed the suite.
…o free RAM" This reverts commit 257f178.
`ccdaManager` was a module-level `await mkCCDARegister()` that loaded the CDA package into memory on test process start and pinned it for the whole run, even though only one describe in typescript.test.ts uses it. Inline the call so CDA is constructed (and reclaimable) inside that describe's scope instead.
`make test` was OOM-killed on ubuntu-latest during SQL-on-FHIR's build: the test fetches the remote tarball and runs the TS + Python + C# generators against R5 + SQL-on-FHIR, and its peak alone — on top of whatever the rest of the suite already had resident — exceeded the runner's ~7 GiB. Split it into a separate `bun test` invocation so its heap starts fresh, independent of the other test files' module-level state.
Even in its own bun process, sql-on-fhir.test.ts was OOM-killed: its
three describes (TypeScript / Python / C#) each built a full generator
output against R5 + the SQL-on-FHIR IG, and the three results held by
closure simultaneously exceeded the ubuntu-latest runner's ~7 GiB.
Split into three per-language files (matching the layout of the
existing typescript.test.ts / python.test.ts / csharp.test.ts unit
tests) and run each as a separate bun invocation in the Makefile, so
peak memory is bounded to one generator's output at a time.
Snapshots split alongside; describe paths are flat
("SQL-on-FHIR <lang> Generation") so snap keys are unchanged.
`oven-sh/setup-bun@v2` defaults to latest stable. Bun 1.3.14 introduced a memory regression that OOM-kills `make test` on `ubuntu-latest`; the same suite passed in 62s on 1.3.13. Pin via `.bun-version` (read by setup-bun automatically) so CI and local dev share the known-good version. Also document the SQL-on-FHIR per-language test split in CLAUDE.md so the structure is not "cleaned up" back into one file.
The bare `oven-sh/setup-bun@v2` step does NOT auto-detect .bun-version; the previous push still ran Bun 1.3.14. Add explicit `bun-version-file: .bun-version` to every setup-bun invocation so the file actually pins the version.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The
Canary & Releasejob has been OOM-killed (make: *** [Makefile:34: test] Killed, exit 143) onubuntu-latestduringtest/api/write-generator/multi-package/sql-on-fhir.test.ts. The kernel SIGKILLsbun testonce resident memory exceeds the runner's ~7 GiB. Re-runs are deterministic-fail.Root cause
All three multi-package test files use top-level
awaitinsidedescribe:Two compounding problems:
describeresolve during discovery, so every section'sresultis built upfront — three (or four) results live concurrently per file.resultis captured by everyit()in the section, so GC can't reclaim it for the file's lifetime.Each
result.filesGeneratedis aRecord<string, string>of full file contents (CDA TS: 124 files, SQL-on-FHIR + R5 deps: 45+ R5 files plus generators × 3). Withbunfig.tomlsettingparallel = true, multiple files' worth of residency pile up at the same time.Fix
Move generation into
beforeAlland null the result inafterAll(+Bun.gc(true)) so each section's payload can be reclaimed before the next section starts:Applied to:
test/api/write-generator/multi-package/cda.test.tstest/api/write-generator/multi-package/local-package.test.tstest/api/write-generator/multi-package/sql-on-fhir.test.tsVerification
bun run typecheck— cleanbun run lint— only pre-existing warnings insrc/typeschema/utils.tsbun test— 257 pass, 0 fail, 75 snapshots intactubuntu-latest, so CI on this PR is the real check.