From 281e0a05fdee4b9a12d0eec1bb7050927f2fc0ed Mon Sep 17 00:00:00 2001 From: Jan Grodowski Date: Thu, 11 Jun 2026 10:29:30 +0200 Subject: [PATCH 1/2] Document missing command-line flags. Add sections for command-line flags that were registered in the CLI but missing from command-line-flags.md, including logging, throttling, hooks, StatsD, trigger, revert, and basic connection flags. Update the documented cut-over default from the stale "safe" value to the current "atomic" default. --- doc/command-line-flags.md | 191 +++++++++++++++++++++++++++++++++++++- 1 file changed, 189 insertions(+), 2 deletions(-) diff --git a/doc/command-line-flags.md b/doc/command-line-flags.md index 2012c8c4c..7a56b2c05 100644 --- a/doc/command-line-flags.md +++ b/doc/command-line-flags.md @@ -10,7 +10,11 @@ Add this flag when executing on Aliyun RDS. Allows the user to make schema changes that include a zero date or zero in date (e.g. adding a `datetime default '0000-00-00 00:00:00'` column), even if global `sql_mode` on MySQL has `NO_ZERO_IN_DATE,NO_ZERO_DATE`. -### azure +### alter + +Mandatory unless using [`--revert`](#revert). The schema change to apply to the migrated table. You can pass just the alter options, for example `--alter="ADD COLUMN created_at timestamp NULL"`, together with [`--database`](#database) and [`--table`](#table). You can also pass a full `ALTER TABLE [database.]table ...` statement; an explicit database or table in `--alter` can be used instead of the corresponding flag. + +### azure Add this flag when executing on Azure Database for MySQL. @@ -18,6 +22,10 @@ Add this flag when executing on Azure Database for MySQL. See [`--assume-master-host`](#assume-master-host). +### allow-nullable-unique-key + +Allows `gh-ost` to choose a unique key that contains nullable columns. This is dangerous if the chosen key contains `NULL` values: row identity may be ambiguous and data may be corrupted. Use only when you know the nullable unique key contains no `NULL` values. + ### allow-on-master By default, `gh-ost` would like you to connect to a replica, from where it figures out the master by itself. This wiring is required should your master execute using `binlog_format=STATEMENT`. @@ -37,6 +45,10 @@ When your migration issues a column rename (`change column old_name new_name ... If you think `gh-ost` is mistaken and that there's actually no _rename_ involved, you may pass [`--skip-renamed-columns`](#skip-renamed-columns) instead. This will cause `gh-ost` to disassociate the column values; data will not be copied between those columns. +### ask-pass + +Prompt for the MySQL password instead of passing it on the command line via [`--password`](#password). + ### assume-master-host `gh-ost` infers the identity of the master server by crawling up the replication topology. You may explicitly tell `gh-ost` the identity of the master host via `--assume-master-host=the.master.com`. This is useful in: @@ -69,6 +81,12 @@ It is not reliable to parse the `ALTER` statement to determine if it is instant ### binlogsyncer-max-reconnect-attempts `--binlogsyncer-max-reconnect-attempts=0`, the maximum number of attempts to re-establish a broken inspector connection for sync binlog. `0` or `negative number` means infinite retry, default `0` +### check-flag + +Checks whether one or more flags exist in the current `gh-ost` binary. This is useful for cross-version scripting. Exits with `0` when all additionally provided flags exist, non-zero otherwise. Flags that require a value must be given a dummy value. + +Example: `gh-ost --check-flag --cut-over-lock-timeout-seconds 1 --nice-ratio 0` + ### checkpoint `--checkpoint` enables periodic checkpoints of the gh-ost's state so that gh-ost can resume a migration from the checkpoint with `--resume`. Checkpoints are written to a separate table named `_${original_table_name}_ghk`. It is recommended to use with `--gtid` for checkpoints. @@ -78,6 +96,10 @@ See also: [`resuming-migrations`](resume.md) `--checkpoint-seconds` specifies the seconds between checkpoints. Default is 300. +### chunk-size + +Controls how many rows `gh-ost` copies in each row-copy iteration. The default is `1000`; allowed range is `10` to `100000`. + ### conf `--conf=/path/to/my.cnf`: file where credentials are specified. Should be in (or contain) the following format: @@ -114,12 +136,28 @@ This is somewhat similar to a Nagios `n`-times test, where `n` in our case is al ### cut-over -Optional. Default is `safe`. See more discussion in [`cut-over`](cut-over.md) +Optional. Default is `atomic`. Accepted values are `atomic`, `default`, and `two-step`. See more discussion in [`cut-over`](cut-over.md) + +### cut-over-exponential-backoff + +Wait exponentially longer intervals between failed cut-over attempts. The maximum wait interval is controlled by [`--exponential-backoff-max-interval`](#exponential-backoff-max-interval). ### cut-over-lock-timeout-seconds Default `3`. Max number of seconds to hold locks on tables while attempting to cut-over (retry attempted when lock exceeds timeout). +### database + +The database containing the table to migrate. Required unless the database is specified in a full [`--alter`](#alter) statement. + +### debug + +Enables debug logging. This is very verbose. See also [`--verbose`](#verbose), [`--quiet`](#quiet), and [`--stack`](#stack). + +### default-retries + +Default number of retries for various operations before panicking. Default is `60`. + ### discard-foreign-keys **Danger**: this flag will _silently_ discard any foreign keys existing on your table. @@ -158,6 +196,10 @@ While the ongoing estimated number of rows is still heuristic, it's almost exact Without this parameter, migration is a _noop_: testing table creation and validity of migration, but not touching data. +### exponential-backoff-max-interval + +Maximum number of seconds to wait between attempts when exponential backoff is used. Default is `64` seconds. See also [`--cut-over-exponential-backoff`](#cut-over-exponential-backoff). + ### force-named-cut-over If given, a `cut-over` command must name the migrated table, or else ignored. @@ -182,6 +224,30 @@ Add this flag to enable support for [MySQL replication GTIDs](https://dev.mysql. Default 100. See [`subsecond-lag`](subsecond-lag.md) for details. +### help + +Prints command-line usage and exits. + +### host + +MySQL hostname to connect to. Preferably this is a replica, not the master. Default is `127.0.0.1`. + +### hooks-hint + +Arbitrary message injected into hooks via the `GH_OST_HOOKS_HINT` environment variable. + +### hooks-hint-owner + +Arbitrary owner name injected into hooks via the `GH_OST_HOOKS_HINT_OWNER` environment variable. + +### hooks-hint-token + +Arbitrary token injected into hooks via the `GH_OST_HOOKS_HINT_TOKEN` environment variable. + +### hooks-path + +Directory where hook files are found. Empty by default, which disables hooks. Hook files found on this path and conforming to hook naming conventions are executed. + ### hooks-status-interval Defaults to 60 seconds. Configures how often the `gh-ost-on-status` hook is called, see [`hooks`](hooks.md) for full details on how to use hooks. @@ -200,6 +266,22 @@ See [`initially-drop-ghost-table`](#initially-drop-ghost-table) Default False. Should `gh-ost` forcibly delete an existing socket file. Be careful: this might drop the socket file of a running migration! +### ignore-http-errors + +Ignore HTTP connection errors during checks configured with [`--throttle-http`](#throttle-http). + +### include-triggers + +When true, existing triggers on the original table are created on the new table. See also [`--trigger-suffix`](#trigger-suffix) and [`--remove-trigger-suffix-if-exists`](#remove-trigger-suffix-if-exists). + +### master-password + +MySQL password for the master when it differs from the password used for the inspected replica. Requires [`--assume-master-host`](#assume-master-host). + +### master-user + +MySQL user for the master when it differs from the user used for the inspected replica. Requires [`--assume-master-host`](#assume-master-host). + ### max-lag-millis On a replication topology, this is perhaps the most important migration throttling factor: the maximum lag allowed for migration to work. If lag exceeds this value, migration throttles. @@ -220,18 +302,50 @@ List of metrics and threshold values; topping the threshold of any will cause th Typically `gh-ost` is used to migrate tables on a master. If you wish to only perform the migration in full on a replica, connect `gh-ost` to said replica and pass `--migrate-on-replica`. `gh-ost` will briefly connect to the master but otherwise will make no changes on the master. Migration will be fully executed on the replica, while making sure to maintain a small replication lag. +### mysql-timeout + +Connect, read, and write timeout for MySQL connections. The default `0` uses the driver's default behavior. + +### nice-ratio + +Makes row-copy operations sleep after each chunk, proportional to the time spent copying that chunk. Range is `0.0` to `100.0`; default `0` is most aggressive. For example, `--nice-ratio=1` sleeps one millisecond for every millisecond spent copying rows. + +### ok-to-drop-table + +Allows `gh-ost` to drop the old table at the end of a successful migration. This is disabled by default because dropping a table can be a long locking operation. + +### old-table + +The old table name to use when [`--revert`](#revert) is enabled, for example `_mytable_del`. + +### panic-flag-file + +When this file is created, `gh-ost` immediately terminates without cleanup. + ### panic-on-warnings When this flag is set, `gh-ost` will panic when SQL warnings indicating data loss are encountered when copying data. This flag helps prevent data loss scenarios with migrations touching unique keys, column collation and types, as well as `NOT NULL` constraints, where `MySQL` will silently drop inserted rows that no longer satisfy the updated constraint (also dependent on the configured `sql_mode`). While `panic-on-warnings` is currently disabled by defaults, it will default to `true` in a future version of `gh-ost`. +### password + +MySQL password. To avoid passing the password on the command line, use [`--ask-pass`](#ask-pass) or [`--conf`](#conf). + +### port + +MySQL port to connect to. Preferably this is a replica, not the master. Default is `3306`. + ### postpone-cut-over-flag-file Indicate a file name, such that the final [cut-over](cut-over.md) step does not take place as long as the file exists. When this flag is set, `gh-ost` expects the file to exist on startup, or else tries to create it. `gh-ost` exits with error if the file does not exist and `gh-ost` is unable to create it. With this flag set, the migration will cut-over upon deletion of the file or upon `cut-over` [interactive command](interactive-commands.md). +### quiet + +Quiet mode. Forces logging to errors only, overriding [`--verbose`](#verbose) or [`--debug`](#debug) if they are also provided. + ### replica-server-id Defaults to 99999. If you run multiple migrations then you must provide a different, unique `--replica-server-id` for each `gh-ost` process. @@ -240,14 +354,35 @@ Optionally involve the process ID, for example: `--replica-server-id=$((10000000 It's on you to choose a number that does not collide with another `gh-ost` or another running replica. See also: [`concurrent-migrations`](cheatsheet.md#concurrent-migrations) on the cheatsheet. +### remove-trigger-suffix-if-exists + +Remove the given suffix from trigger names. Requires [`--include-triggers`](#include-triggers) and [`--trigger-suffix`](#trigger-suffix). + +### replication-lag-query + +Deprecated. `gh-ost` uses an internal subsecond-resolution lag query instead. + ### resume `--resume` attempts to resume a migration that was previously interrupted from the last checkpoint. The first `gh-ost` invocation must run with `--checkpoint` and have successfully written a checkpoint in order for `--resume` to work. See also: [`resuming-migrations`](resume.md) +### revert + +Attempts to revert a completed migration. Must be used with [`--old-table`](#old-table). `--revert` cannot be used together with [`--resume`](#resume), and ignores migration options such as `--alter`, `--attempt-instant-ddl`, `--include-triggers`, and `--discard-foreign-keys`. + +### runtime-metrics-interval + +Seconds between Go runtime memory/GC gauge samples. Requires [`--statsd-addr`](#statsd-addr). Default is `10`; `0` disables runtime metrics. + ### serve-socket-file Defaults to an auto-determined and advertised upon startup file. Defines Unix socket file to serve on. + +### serve-tcp-port + +TCP port for `gh-ost` to serve its interactive interface on. Default is disabled. + ### skip-foreign-key-checks By default `gh-ost` verifies no foreign keys exist on the migrated table. On servers with large number of tables this check can take a long time. If you're absolutely certain no foreign keys exist (table does not reference other table nor is referenced by other tables) and wish to save the check time, provide with `--skip-foreign-key-checks`. @@ -259,6 +394,10 @@ By default `gh-ost` performs a check before the cut-over to ensure the rename se See also: [`allow-setup-metadata-lock-instruments`](#allow-setup-metadata-lock-instruments) +### skip-port-validation + +Skips port validation for MySQL connections. + ### skip-strict-mode By default `gh-ost` enforces STRICT_ALL_TABLES sql_mode as a safety measure. In some cases this changes the behaviour of other modes (namely ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE) which may lead to errors during migration. Use `--skip-strict-mode` to explicitly tell `gh-ost` not to enforce this. **Danger** This may have some unexpected disastrous side effects. @@ -287,6 +426,18 @@ Allows `gh-ost` to connect to the MySQL servers using encrypted connections, but `--ssl-key=/path/to/ssl-key.key`: SSL private key file (in PEM format). +### stack + +Adds a stack trace when logging errors. + +### statsd-addr + +StatsD endpoint, either `host:port` or a Unix socket. Empty disables StatsD. + +### statsd-tags + +Global StatsD tags applied to every metric. This flag is repeatable and uses `key:value` format. Example: `--statsd-tags 'env:prod' --statsd-tags 'service:gh-ost'`. + ### storage-engine Default is `innodb`, and `rocksdb` support is currently experimental. InnoDB and RocksDB are both transactional engines, supporting both shared and exclusive row locks. @@ -302,6 +453,14 @@ When `--storage-engine=rocksdb`, `gh-ost` will make some changes necessary (e.g. ### charset The default charset for the database connection is utf8mb4, utf8, latin1. The ability to specify character set and collation is supported, eg: utf8mb4_general_ci,utf8_general_ci,latin1. +### switch-to-rbr + +Allows `gh-ost` to automatically switch the replica's binary log format to `ROW`, if needed. The format is not switched back automatically. + +### table + +The table name to migrate, without the database name. Required unless the table is specified in a full [`--alter`](#alter) statement. See also [`--database`](#database). + ### test-on-replica Issue the migration on a replica; do not modify data on master. Useful for validating, testing and benchmarking. See [`testing-on-replica`](testing-on-replica.md) @@ -310,6 +469,18 @@ Issue the migration on a replica; do not modify data on master. Useful for valid Default `False`. When `--test-on-replica` is enabled, do not issue commands stop replication (requires `--test-on-replica`). +### throttle-additional-flag-file + +An additional flag file that pauses operation while it exists. Defaults to `/tmp/gh-ost.throttle` and is useful for throttling multiple `gh-ost` operations at once. + +### throttle-flag-file + +A flag file that pauses operation while it exists. Prefer a file name specific to the table being altered. + +### throttle-query + +A custom query issued every second to decide whether the migration should throttle. The query runs on the migrated server and should return `0` for no throttle and a value greater than `0` to throttle. Keep this query lightweight. + ### throttle-control-replicas Provide a command delimited list of replicas; `gh-ost` will throttle when any of the given replicas lag beyond [`--max-lag-millis`](#max-lag-millis). The list can be queried and updated dynamically via [interactive commands](interactive-commands.md) @@ -330,6 +501,22 @@ Defaults to 1000 (1 second). Configures the HTTP throttler check timeout in mill Makes the _old_ table include a timestamp value. The _old_ table is what the original table is renamed to at the end of a successful migration. For example, if the table is `gh_ost_test`, then the _old_ table would normally be `_gh_ost_test_del`. With `--timestamp-old-table` it would be, for example, `_gh_ost_test_20170221103147_del`. +### trigger-suffix + +Adds a suffix to trigger names, for example `_v2`. Requires [`--include-triggers`](#include-triggers). + ### tungsten See [`tungsten`](cheatsheet.md#tungsten) on the cheatsheet. + +### user + +MySQL user. + +### verbose + +Enables info-level logging. See also [`--debug`](#debug) and [`--quiet`](#quiet). + +### version + +Prints the `gh-ost` version and exits. From 19363dffcee688046f32ff45bdcb53f9366a1511 Mon Sep 17 00:00:00 2001 From: Jan Grodowski Date: Thu, 11 Jun 2026 11:01:42 +0200 Subject: [PATCH 2/2] Adress AI review --- doc/command-line-flags.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/command-line-flags.md b/doc/command-line-flags.md index 7a56b2c05..d017c7c25 100644 --- a/doc/command-line-flags.md +++ b/doc/command-line-flags.md @@ -83,7 +83,7 @@ It is not reliable to parse the `ALTER` statement to determine if it is instant ### check-flag -Checks whether one or more flags exist in the current `gh-ost` binary. This is useful for cross-version scripting. Exits with `0` when all additionally provided flags exist, non-zero otherwise. Flags that require a value must be given a dummy value. +Checks whether one or more flags exist in the current `gh-ost` binary. This is useful for cross-version scripting. Exits with `0` when all flags specified alongside `--check-flag` exist, non-zero otherwise. Flags that require a value must be given a dummy value. Example: `gh-ost --check-flag --cut-over-lock-timeout-seconds 1 --nice-ratio 0` @@ -136,7 +136,7 @@ This is somewhat similar to a Nagios `n`-times test, where `n` in our case is al ### cut-over -Optional. Default is `atomic`. Accepted values are `atomic`, `default`, and `two-step`. See more discussion in [`cut-over`](cut-over.md) +Optional. Default is `atomic`. Accepted values are `atomic`, `default`, and `two-step`; `default` is a legacy alias for `atomic`. `atomic` performs an atomic table swap where pending connections briefly block and either all see the old table or all see the new table. `two-step` uses a non-atomic rename sequence where the original table is renamed away before the ghost table takes its place, leaving a brief interval where the table does not exist. See more discussion in [`cut-over`](cut-over.md) ### cut-over-exponential-backoff @@ -304,7 +304,7 @@ Typically `gh-ost` is used to migrate tables on a master. If you wish to only pe ### mysql-timeout -Connect, read, and write timeout for MySQL connections. The default `0` uses the driver's default behavior. +Connect, read, and write timeout for MySQL connections, in seconds. The value is applied to the MySQL driver's `timeout`, `readTimeout`, and `writeTimeout` DSN parameters. It is configured on the initial inspector connection and is copied to related gh-ost MySQL connections, including the applier/master connection and throttle-control replica checks. The default `0` uses the driver's default behavior. ### nice-ratio