Skip to content

perf(test): clone a migrated template DB per TestD1Database instead of replaying 178 migrations#8533

Merged
JSONbored merged 1 commit into
mainfrom
perf/test-d1-template-clone
Jul 24, 2026
Merged

perf(test): clone a migrated template DB per TestD1Database instead of replaying 178 migrations#8533
JSONbored merged 1 commit into
mainfrom
perf/test-d1-template-clone

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Every TestD1Database() construction executed the full migrations/*.sql chain (178 files, growing with every merge) into a fresh :memory: database — measured at ~640ms per construction, across ~1,500 construction sites: ~950s of pure schema replay per full run, and the reason ordinary tests sat in the 3s+ range (observed live on PR #8524's validate-tests).

This replaces the per-construction replay with a content-hash-keyed, fully-migrated template database file built once per migration-chain content and shared across all vitest workers; each TestD1Database gets its own copyFileSync clone (~1.5ms including a verified write — ~440× faster per construction).

Measured on the full unsharded suite (same machine, same tree):

before after
wall time 510s 171s (3.0×)
aggregate test time 4,288s 870s (4.9×)
heaviest file (queue.test.ts) 107s 17s

Design constraints the shape answers (each verified the hard way, documented in the helper):

  • serialize()/deserialize() don't exist on pinned Node 22 (the prior attempt crashed CI) — file copy is Node-22-safe.
  • Clones must NOT be unlinked while open: SQLite fails later writes with SQLITE_READONLY_DBMOVED. Clones stay on disk until a single per-process exit sweep.
  • Vitest's per-file module isolation re-evaluates the helper constantly, so all memoization lives on globalThis — module-scope memos would rebuild the template once per test file and silently give back most of the win.
  • Template built at a .tmp sibling then atomically renameSyncd — concurrent workers double-build harmlessly; no reader ever copies a half-written template; a schema change gets a fresh hash key.
  • page_size=1024 + VACUUM shrink the empty-schema template 1.4MB → ~410KB, bounding worst-case tmpdir usage for a run's clones to a few hundred MB, swept at exit.

Validation

  • npm run test:coverage (full, unsharded): 21,472 passed / 0 failed — identical pass set to the pre-change baseline run, at 171s vs 510s.
  • Only test/helpers/d1.ts changes (Codecov-ignored path); no runtime code touched.

…f replaying 178 migrations

Executing the full migrations/*.sql chain into :memory: on every construction
(~640ms x ~1500 sites) was ~950s of pure schema replay per full run and put
ordinary tests in the 3s+ range. Build one content-hash-keyed template file per
migration-chain content (atomic rename, shared across workers, page_size=1024 +
VACUUM), then copyFileSync a clone per instance: ~1.5ms per construction with a
verified write. Full suite: 510s -> 171s wall, 4288s -> 870s aggregate, same
21472-test pass set.

Clones stay on disk until a single exit sweep -- SQLite fails writes with
SQLITE_READONLY_DBMOVED if the main file is unlinked while open -- and all
memoization lives on globalThis so vitest's per-file module isolation cannot
silently rebuild the template per test file. serialize()/deserialize() remain
unusable on the pinned Node 22 (the prior attempt crashed CI).
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 24, 2026
@JSONbored
JSONbored merged commit c3d9277 into main Jul 24, 2026
3 of 4 checks passed
@JSONbored
JSONbored deleted the perf/test-d1-template-clone branch July 24, 2026 19:01
@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
21448 3 21445 21
View the top 3 failed test(s) by shortest run time
test/unit/rag-index.test.ts > indexRepo: MAX_CHUNKS_PER_REPO cap holds > still indexes package.json (and other root manifest/config files) on a repo whose file count exceeds the cap (regression: manifestPriority)
Stack Traces | 36s run time
Error: Test timed out in 15000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
 ❯ test/unit/rag-index.test.ts:530:3
test/unit/rag-index.test.ts > indexRepo: MAX_CHUNKS_PER_REPO cap holds > stops upserting once the per-repo cap is reached
Stack Traces | 37.2s run time
Error: Test timed out in 15000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
 ❯ test/unit/rag-index.test.ts:511:3
test/unit/rag-index.test.ts > reindexChangedPaths: delete + re-upsert only the changed paths > caps incremental reindex upserts at MAX_CHUNKS_PER_REPO
Stack Traces | 43.8s run time
Error: Test timed out in 15000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".
 ❯ test/unit/rag-index.test.ts:735:3

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant