feat: add database migration system with clean testing environment

[jumski]

Add Database Migration System for Edge Worker

This PR introduces a database migration system for the Edge Worker, enabling automated schema management through a structured approach. Key features include:

• Created a MigrationRunner class that handles:

  • Creating the pgflow_installer schema and tracking table
  • Listing migration status (pending/applied)
  • Applying pending migrations with advisory locking for concurrency safety

• Added new API endpoints to the control plane:

  • GET /migrations/list - Shows all migrations with their status
  • POST /migrations/up - Applies pending migrations

• Implemented a migration loader that reads SQL files from the @pgflow/core package

• Added comprehensive tests for the migration system with a dedicated clean database setup:

  • Created Docker Compose configuration for a clean test database
  • Added scripts to ensure database is properly initialized
  • Wrote tests verifying migration listing, application, and idempotency

• Updated Supabase JS dependency from 2.84.0 to 2.86.0

The migration system ensures database schema changes can be applied safely and consistently across environments.

[changeset-bot]

🦋 Changeset detected

Latest commit: 7caca1d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
@pgflow/edge-worker	Patch
@pgflow/core	Patch
@pgflow/dsl	Patch
@pgflow/client	Patch
pgflow	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[jumski]

• feat: add Installer for no-CLI platforms like Lovable #551
• feat: add database migration system with clean testing environment #544 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[nx-cloud]

View your CI Pipeline Execution ↗ for commit 7caca1d

Command	Status	Duration	Result
nx run edge-worker:test:integration	✅ Succeeded	3m 53s	View ↗
nx run client:e2e	✅ Succeeded	1m 11s	View ↗
nx run core:pgtap	✅ Succeeded	1m 37s	View ↗
nx affected -t verify-exports --base=origin/mai...	✅ Succeeded	3s	View ↗
nx run cli:e2e	✅ Succeeded	5s	View ↗
nx affected -t build --configuration=production...	✅ Succeeded	4s	View ↗
nx affected -t lint typecheck test --parallel -...	✅ Succeeded	1m	View ↗
nx run edge-worker:e2e	✅ Succeeded	49s	View ↗

---

☁️ Nx Cloud last updated this comment at 2025-12-22 10:38:53 UTC

[graphite-app]

Race condition handling bug causes incorrect statistics. When another process applies a migration while this process waits for the lock, the code returns early at line 68 but execution continues after the transaction block (line 81-82), incorrectly incrementing the applied counter and marking the migration as 'applied' in results.

This causes the applied count in the final result to be wrong - it counts migrations that were actually applied by other processes.

Fix:
Track whether the migration was actually applied in this transaction:

let wasApplied = false;
await this.sql.begin(async (tx) => {
  await tx`SELECT pg_advisory_xact_lock(${MIGRATION_LOCK_KEY})`;
  
  const existing = await tx`
    SELECT 1 FROM pgflow_installer.migrations
    WHERE timestamp = ${migration.timestamp}
  `;
  
  if (existing.length > 0) {
    return; // Already applied by another process
  }
  
  await tx.unsafe(migration.content);
  await tx`
    INSERT INTO pgflow_installer.migrations (timestamp)
    VALUES (${migration.timestamp})
  `;
  wasApplied = true;
});

if (wasApplied) {
  results.push({ timestamp: migration.timestamp, status: 'applied' });
  applied++;
} else {
  results.push({ timestamp: migration.timestamp, status: 'skipped' });
  skipped++;
}

Suggested change

	if (existing.length > 0) {
	// Another process applied it while we were waiting for lock
	return;
	}
	// Execute migration SQL
	await tx.unsafe(migration.content);
	// Record that migration was applied
	await tx`
	INSERT INTO pgflow_installer.migrations (timestamp)
	VALUES (${migration.timestamp})
	`;
	});
	results.push({ timestamp: migration.timestamp, status: 'applied' });
	applied++;
	if (existing.length > 0) {
	// Another process applied it while we were waiting for lock
	return;
	}
	// Execute migration SQL
	await tx.unsafe(migration.content);
	// Record that migration was applied
	await tx`
	INSERT INTO pgflow_installer.migrations (timestamp)
	VALUES (${migration.timestamp})
	`;
	// Flag that we applied this migration
	wasApplied = true;
	});
	if (wasApplied) {
	results.push({ timestamp: migration.timestamp, status: 'applied' });
	applied++;
	} else {
	results.push({ timestamp: migration.timestamp, status: 'skipped' });
	skipped++;
	}

Spotted by Graphite Agent



Is this helpful? React 👍 or 👎 to let us know.

[github-actions]

🔍 Preview Deployment: Website

✅ Deployment successful!

🔗 Preview URL: https://pr-544.pgflow.pages.dev

📝 Details:

• Branch: 12-11-add_migration_installer_api_endpoints_for_no-cli_platforms
• Commit: 394148edec1a196641466120b669f7632d9d5d60
• View Logs

_Last updated: _