fix: clean up migration file packaging using Bun.embeddedFiles#155
Merged
fix: clean up migration file packaging using Bun.embeddedFiles#155
Conversation
PR #146 embedded SQL migration files as bun binary assets and read them back via Bun.embeddedFiles at runtime — a brittle approach that required glob patterns and --asset-naming in every compile script. This replaces that with make-vfs (seveibar/make-vfs): - Add `generate:migrations-vfs` script that runs make-vfs on both drizzle/migrations (SQLite) and drizzle/migrations_pg (Postgres), emitting migrations-vfs-sqlite.ts and migrations-vfs-pg.ts as standard TypeScript modules with SQL content as decoded strings. - migrate.ts now imports those VFS modules plus the _journal.json files directly. getMigrationsDir() writes VFS content to a per-process tmpdir and returns the path for drizzle's migrator — no conditional Bun.embeddedFiles detection, no BunEmbeddedFile interface. - Compile scripts (compile:linux/macos/windows, build:bin) revert to the pre-PR#146 form: only TS entry point + frontend HTML/JS/CSS. No SQL globs, no --asset-naming flag. The VFS .ts files are bundled by Bun as ordinary TypeScript. - Delete migrations-bundle.ts (no longer needed). To regenerate the VFS after drizzle-kit generates new migrations, run: bun run generate:migrations-vfs https://claude.ai/code/session_01J5ev59zvC8RqRifVxi2FRL
Previous commits only excluded .test-db.sqlite-shm; add the main DB file and WAL file so running `bun test` no longer produces uncommitted changes. https://claude.ai/code/session_01J5ev59zvC8RqRifVxi2FRL
Instead of writing VFS content to a temp directory so drizzle-orm's migrate() can read it back from disk, build the MigrationMeta[] array that drizzle's internal dialect.migrate() expects and call it directly. drizzle-orm's migrate() is a two-step wrapper: 1. readMigrationFiles(config) — reads folder → MigrationMeta[] 2. db.dialect.migrate(migrations, session, config) — applies to DB migrationsFromVfs() replicates step 1 from VFS strings instead of fs, then we call step 2 directly. drizzle still owns all migration tracking logic (__drizzle_migrations table creation, hash recording, skip-if-applied). Removes: os, fs, path imports; getMigrationsDir(); migrationsDirCache; tmpdir writes; process exit cleanup handlers. attemptPostgresDuplicateColumnRepair() now reads from pgVfs/pgJournal directly instead of from the (now-gone) migrationsPath on disk. https://claude.ai/code/session_01J5ev59zvC8RqRifVxi2FRL
Remove make-vfs entirely. SQL migration files are embedded in the compiled binary via glob patterns in the compile scripts (same as before PR #146), and accessed at runtime via Bun.embeddedFiles. The key difference from PR #146's implementation: - No path-matching regex to find files — we use the journal to know exactly which filenames to expect, then do a direct Map lookup - Journal JSON is imported directly (not via embeddedFiles) since Bun transpiles JSON to a JS module rather than embedding it as a file asset - In dev/source mode, embeddedFiles is empty so readSql() falls back to Bun.file() at the import.meta.dir-relative filesystem path — no conditional binary-detection logic needed, just a Map lookup miss compile scripts: restore SQL globs + --asset-naming="[name].[ext]", drop the generate:migrations-vfs prebuild step entirely. migrate.ts still calls db.dialect.migrate() directly with a built MigrationMeta[] (no folder-based migrate() wrapper). 845/845 tests passing. https://claude.ai/code/session_01J5ev59zvC8RqRifVxi2FRL
Log the source of migrations (embedded or filesystem) and the count so startup output confirms which path was taken. https://claude.ai/code/session_01J5ev59zvC8RqRifVxi2FRL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR #146 introduced a fragile approach to embedding drizzle migration files in the compiled binary: SQL files were listed individually as glob patterns in every compile script,
--asset-naming="[name].[ext]"was required, andmigrate.tsused complex path-regex filtering overBun.embeddedFilesto find them at runtime. The journal JSON required a completely separate code path (migrations-bundle.ts) because Bun transpiles JSON to JS rather than treating it as a file asset.This PR replaces all of that with a clean, minimal implementation.
What changed
migrate.ts— complete rewrite of the migration loading path:Bun.embeddedFilesis loaded into aMap<name, blob>once at module loadreadSql(tag, devDir)does a direct map lookup by${tag}.sql; if the map is empty (dev/source mode), falls back toBun.file()at theimport.meta.dir-relative filesystem path — no binary-detection logic, just a map missbuildMigrations()assembles theMigrationMeta[]array thatdb.dialect.migrate()expects, bypassing drizzle's folder-basedmigrate()wrapper entirely — drizzle still owns all tracking-table logicattemptPostgresDuplicateColumnRepairuses the already-loadedMigrationMeta[]instead of re-reading from diskembeddedvsfilesystem) and count at startup so it's always clear which path ranpackage.json:make-vfsdevDependency (added mid-branch, removed again after finding a better approach)generate:migrations-vfsscript--asset-naming="[name].[ext]"Deleted:
migrations-bundle.ts,migrations-vfs-sqlite.ts,migrations-vfs-pg.ts.gitignore: added root-level.test-db.sqliteentriesVerified
plexus-linuxbinary started against a fresh SQLite DB and logged:Bun.embeddedFilesis the source in compiled mode, not the filesystem fallback.