Move Tables: 1.2 skip-tables and run overall move tables flow except cutover (#8206)

[zacharysierakowski]

closes https://github.com/github/database-infrastructure/issues/8206

Building on the work from @chriskirkland and @womoruyi over in #1700, this PR just takes the code from that branch and patches up the git history so there aren't conflicts against the feature branch for the move-tables effort.

I also took some time to clean up some lingering logging from the demo. And applied some changes to fix up some leftover TODO(chriskirkland)s 🙂

For full PR description, i'm deferring to #1700 because there's already a ton of context there!

[danieljoos]

Nice 🚀

[Copilot]

Pull request overview

This PR continues productionizing --move-tables mode by wiring the “skip ghost/changelog/heartbeat” behavior into the main migrator/applier flow, adding local docker-based move-tables helper scripts, and updating tests to reflect the new move-tables control paths.

Changes:

• Add GetTargetDatabaseName() / GetTargetTableName() helpers and thread move-tables-aware naming/connection handling through migrator, inspector, and applier flows.
• Disable changelog listener/heartbeat (and other ghost/changelog behaviors) in move-tables mode, and adjust row-copy to read from an explicit source DB when moving across clusters.
• Add local script/move-tables/* and localtests/docker-compose-move-tables.yml scaffolding plus related tests.

Show a summary per file

File	Description
go/logic/applier.go	Move-tables target DB/table handling, changelog no-op, target-table creation, and row-copy API changes.
go/logic/migrator.go	Integrates move-tables streaming/copy flow and gates throttling/heartbeat/changelog usage.
go/logic/inspect.go	Makes “original table name” move-tables-aware and factors unique-key selection.
go/base/context.go	Adds target DB/table helpers; derives move-tables connection config and applies TLS to it.
go/cmd/gh-ost/main.go	Initializes move-tables connection config and reorders connection config setup.
go/logic/applier_test.go	Adds/adjusts tests for move-tables skip behavior and new method signatures.
go/logic/migrator_test.go	Updates tests to pass the new ReadMigrationRangeValues(nil) signature.
go/base/context_test.go	Tests move-tables credential derivation and TLS application.
script/move-tables/*	Adds local setup/teardown helpers and mysql client wrappers for move-tables topology.
localtests/docker-compose-move-tables.yml	Adds a 2-cluster docker-compose topology for local move-tables runs.
localtests/move-tables/create.sql	Seeds a sample table/data set for local move-tables testing.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

go/logic/applier.go:180

• InitDBConnections initializes moveTablesTargetDB twice: the two consecutive if apl.moveTablesConnectionConfig != nil { ... } blocks are identical. This duplicates DB creation/validation work and can leak extra connections if the first succeeds and the second fails later in the function.

	if apl.moveTablesConnectionConfig != nil {
		moveTablesURI := apl.moveTablesConnectionConfig.GetDBUri(apl.migrationContext.GetTargetDatabaseName()) + "&multiStatements=true"
		if apl.moveTablesTargetDB, _, err = mysql.GetDB(apl.migrationContext.Uuid, moveTablesURI); err != nil {
			return err
		}
		if _, err := base.ValidateConnection(apl.moveTablesTargetDB, apl.moveTablesConnectionConfig, apl.migrationContext, apl.name); err != nil {
			return err
		}
	}
	if apl.moveTablesConnectionConfig != nil {
		moveTablesURI := apl.moveTablesConnectionConfig.GetDBUri(apl.migrationContext.GetTargetDatabaseName()) + "&multiStatements=true"
		if apl.moveTablesTargetDB, _, err = mysql.GetDB(apl.migrationContext.Uuid, moveTablesURI); err != nil {
			return err
		}
		if _, err := base.ValidateConnection(apl.moveTablesTargetDB, apl.moveTablesConnectionConfig, apl.migrationContext, apl.name); err != nil {
			return err
		}
	}

go/logic/applier.go:206

• AcquireMigrationLock connects using GetDBUri(migrationContext.DatabaseName). In move-tables mode, the applier connection points at the target cluster, and DatabaseName may not exist there when --target-database differs, causing lock acquisition to fail before the migration can run. Using GetTargetDatabaseName() here keeps the lock connection consistent with the applier DB selection.

func (apl *Applier) AcquireMigrationLock(ctx context.Context) error {
	lockName := buildMigrationLockName(apl.migrationContext.DatabaseName, apl.originalTableName())

	// Use a dedicated *sql.DB so the pinned connection does not consume a
	// slot in apl.db's small pool (mysql.MaxDBPoolConnections).
	lockURI := apl.connectionConfig.GetDBUri(apl.migrationContext.DatabaseName)
	lockDB, err := gosql.Open("mysql", lockURI)
	if err != nil {

• Files reviewed: 17/17 changed files
• Comments generated: 9

[Copilot]

Copilot's findings

Comments suppressed due to low confidence (1)

go/logic/applier.go:180

• This initializes moveTablesTargetDB twice with the same URI, which can leak the first *sql.DB handle and does redundant connection validation. Remove the duplicated second if apl.moveTablesConnectionConfig != nil { ... } block (or consolidate into a single initialization path).

	if !apl.migrationContext.IsMoveTablesMode() {
		// read target table columns from applier
		if err := apl.readTableColumns(); err != nil {
			return err
		}
	}
	if apl.moveTablesConnectionConfig != nil {
		moveTablesURI := apl.moveTablesConnectionConfig.GetDBUri(apl.migrationContext.GetTargetDatabaseName()) + "&multiStatements=true"
		if apl.moveTablesTargetDB, _, err = mysql.GetDB(apl.migrationContext.Uuid, moveTablesURI); err != nil {
			return err
		}
		if _, err := base.ValidateConnection(apl.moveTablesTargetDB, apl.moveTablesConnectionConfig, apl.migrationContext, apl.name); err != nil {
			return err
		}
	}
	if apl.moveTablesConnectionConfig != nil {
		moveTablesURI := apl.moveTablesConnectionConfig.GetDBUri(apl.migrationContext.GetTargetDatabaseName()) + "&multiStatements=true"
		if apl.moveTablesTargetDB, _, err = mysql.GetDB(apl.migrationContext.Uuid, moveTablesURI); err != nil {
			return err
		}
		if _, err := base.ValidateConnection(apl.moveTablesTargetDB, apl.moveTablesConnectionConfig, apl.migrationContext, apl.name); err != nil {
			return err
		}
	}

• Files reviewed: 17/17 changed files
• Comments generated: 6