diff --git a/content/en/docs/18.0/concepts/query-rewriting.md b/content/en/docs/18.0/concepts/query-rewriting.md index f167eab3d..8a62ac0f7 100644 --- a/content/en/docs/18.0/concepts/query-rewriting.md +++ b/content/en/docs/18.0/concepts/query-rewriting.md @@ -95,6 +95,7 @@ Here is a list of all the system variables that are handled by Vitess and how th | charset | VitessAware | | names | VitessAware | | session_uuid | VitessAware | +| migration_context | VitessAware | | enable_system_settings | VitessAware | | read_after_write_gtid | VitessAware | | read_after_write_timeout | VitessAware | diff --git a/content/en/docs/18.0/reference/programs/vtctl/schema-version-permissions.md b/content/en/docs/18.0/reference/programs/vtctl/schema-version-permissions.md index c0ae4b558..f2a228199 100644 --- a/content/en/docs/18.0/reference/programs/vtctl/schema-version-permissions.md +++ b/content/en/docs/18.0/reference/programs/vtctl/schema-version-permissions.md @@ -149,11 +149,11 @@ Validates that the schema on the primary tablet for shard 0 matches the schema o ### ApplySchema -Applies the schema change to the specified keyspace on all shards. The recommended method of applying schema changes is via [Online DDL](../../../../user-guides/schema-changes/managed-online-schema-changes/), which ensures migrations are non-blocking and keeps replication in sync throughout the operation. --ddl_strategy is used to instruct migrations via vreplication, `gh-ost` or `pt-osc` with optional parameters. -request_context allows the user to specify a custom request context for online DDL migrations. If --skip_preflight, SQL goes directly to shards without going through sanity checks. +Applies the schema change to the specified keyspace on all shards. The recommended method of applying schema changes is via [Online DDL](../../../../user-guides/schema-changes/managed-online-schema-changes/), which ensures migrations are non-blocking and keeps replication in sync throughout the operation. --ddl_strategy is used to instruct migrations via vreplication, `gh-ost` or `pt-osc` with optional parameters. -request_context allows the user to specify a custom request context for online DDL migrations. #### Example -
ApplySchema -- [--wait_replicas_timeout=10s] [--ddl_strategy=+] [--request_context= ] [--skip_preflight] {--sql=<sql> || --sql-file=<filename>} <keyspace>
ApplySchema -- [--wait_replicas_timeout=10s] [--ddl_strategy=#### Flags @@ -162,7 +162,6 @@ Applies the schema change to the specified keyspace on all shards. The recommend | caller_id | string | Effective caller ID used for the operation and should map to an ACL name which grants this identity the necessary permissions to perform the operation (this is only necessary when strict table ACLs are used). | | ddl_strategy | string | Online DDL strategy, compatible with @@ddl_strategy session variable (examples: 'gh-ost', 'pt-osc', 'gh-ost --max-load=Threads_running=100' (default "direct"). | | request_context | string | For Only DDL, optionally supply a custom unique string used as context for the migration(s) in this command. By default a unique context is auto-generated by Vitess. | -| skip_preflight | Boolean | Skip pre-apply schema checks, and directly forward schema change query to shards. Ignored when using Online DDL | | sql | string | A list of semicolon-delimited SQL commands. | | sql-file | string | Identifies the file that contains the SQL commands. This file needs to exist on the server, rather than on the client. | | wait_replicas_timeout | Duration | The amount of time to wait for replicas to receive the schema change via replication (default 10s). | diff --git a/content/en/docs/18.0/user-guides/migration/troubleshooting.md b/content/en/docs/18.0/user-guides/migration/troubleshooting.md index 78ccc854d..48523e07d 100644 --- a/content/en/docs/18.0/user-guides/migration/troubleshooting.md +++ b/content/en/docs/18.0/user-guides/migration/troubleshooting.md @@ -137,9 +137,9 @@ This can be caused by a DDL executed on the source table as by default — contr If you want the same or similar DDL to be applied on the target then you can apply that DDL on the target keyspace and then restart the workflow. For example, using the example above: ```bash -$ vtctlclient ApplySchema -- --ddl_strategy=direct --sql="alter table customer add notes varchar(100) not null" customer +$ vtctldclient ApplySchema --ddl-strategy=direct --sql="alter table customer add notes varchar(100) not null" customer -$ vtctlclient Workflow -- customer.commerce2customer start +$ vtctldclient Workflow customer.commerce2customer start ``` If the tables are not very large or the workflow has not made much progress, you can alternatively `Cancel` the current diff --git a/content/en/docs/18.0/user-guides/schema-changes/advanced-usage.md b/content/en/docs/18.0/user-guides/schema-changes/advanced-usage.md index 434193192..a38be59b9 100644 --- a/content/en/docs/18.0/user-guides/schema-changes/advanced-usage.md +++ b/content/en/docs/18.0/user-guides/schema-changes/advanced-usage.md @@ -33,9 +33,9 @@ Thus, the new migration does not get to execute if an identical previous migrati Usage: ```sh -$ vtctldclient ApplySchema -- --migration_context="1111-2222" --skip_preflight --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce +$ vtctldclient ApplySchema -- --migration_context="1111-2222" --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce -$ vtctldclient ApplySchema -- --migration_context="1111-2222" --skip_preflight --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce +$ vtctldclient ApplySchema -- --migration_context="1111-2222" --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce ``` In the above, the two calls are identical. Specifically, they share the exact same `--migration_context` value of `1111-2222`, and the exact same `--sql`. @@ -49,6 +49,16 @@ Notes: - Continuing the above example: if the new migration is successful, the previous migration remains in `failed` state. - The decision whether to run the migration or to mark it as implicitly `complete` only takes place when the migration is scheduled to run. +The context may also be set via VTGate: + +```sql +mysql> set @@ddl_strategy='vitess --'; +mysql> set @@migration_context='1111-2222'; +mysql> alter table customer add column status int unsigned not null; +``` + +By default, the migration context for an Online DDL issued via VTGate is the value of `@@session_uuid`. If `@@migration_context` is non-empty, then its value is used. + ## Duplicate migration indication You may go one step beyond [duplicate migration detection](#duplicate-migration-detection) by explicitly supplying migration UUIDs such that duplicate migrations are never submitted in the first place. @@ -56,9 +66,9 @@ You may go one step beyond [duplicate migration detection](#duplicate-migration- Consider the following example, note `--uuid_list` flag: ```sh -$ vtctldclient ApplySchema -- --uuid_list "73380089_7764_11ec_a656_0a43f95f28a3" --skip_preflight --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce +$ vtctldclient ApplySchema -- --uuid_list "73380089_7764_11ec_a656_0a43f95f28a3" --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce -$ vtctldclient ApplySchema -- --uuid_list "73380089_7764_11ec_a656_0a43f95f28a3" --skip_preflight --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce +$ vtctldclient ApplySchema -- --uuid_list "73380089_7764_11ec_a656_0a43f95f28a3" --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce ``` Normally Vitess generates a `UUID` for each migration, thus having a new, unique ID per migration. With `-uuid_list`, you can force Vitess into using your own supplied UUID. There cannot be two migrations with the same `UUID`. Therefore, any subsequent submission of a migration with an already existing `UUID` is implicitly discarded. The 2nd call does return the migration `UUID`, but is internally discarded. @@ -86,7 +96,7 @@ The user may submit multiple migrations such that non auto-completes. The user c Consider the following: ```sh -$ vtctldclient ApplySchema --skip_preflight --ddl_strategy='vitess --postpone-completion --allow-concurrent' --sql "alter table customer add column country int not null default 0; alter table order add column shipping_country int not null default 0" commerce +$ vtctldclient ApplySchema --ddl_strategy='vitess --postpone-completion --allow-concurrent' --sql "alter table customer add column country int not null default 0; alter table order add column shipping_country int not null default 0" commerce 29231906_776f_11ec_a656_0a43f95f28a3 3cc4ae0e_776f_11ec_a656_0a43f95f28a3 ``` @@ -98,14 +108,14 @@ A `show vitess_migrations like '29231906_776f_11ec_a656_0a43f95f28a3'` presents When all migrations for the relevant UUIDs show `1` for `ready_to_complete`, the user can then either: ```sh -$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration complete all" commerce +$ vtctldclient ApplySchema --sql "alter vitess_migration complete all" commerce ``` Assuming these are the only migrations awaiting completion, or, explicitly issue a complete for each of the migrations: ```sh -$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration '29231906_776f_11ec_a656_0a43f95f28a3' complete all" commerce -$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' complete all" commerce +$ vtctldclient ApplySchema --sql "alter vitess_migration '29231906_776f_11ec_a656_0a43f95f28a3' complete all" commerce +$ vtctldclient ApplySchema --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' complete all" commerce ``` ## Near instant REVERTs @@ -125,14 +135,14 @@ The use case and workflow is as follows: Consider the following example. We run a 5 hour long migration to drop an index: ```sh -$ vtctldclient ApplySchema --skip_preflight --ddl_strategy='vitess' --sql "alter table customer drop index joined_timestamp_idx" commerce +$ vtctldclient ApplySchema --ddl_strategy='vitess' --sql "alter table customer drop index joined_timestamp_idx" commerce 29231906_776f_11ec_a656_0a43f95f28a3 ``` As soon as the migration completes, we run: ```sh -$ vtctldclient ApplySchema --skip_preflight --ddl_strategy='vitess --postpone-completion --allow-concurrent' --sql "revert vitess_migration '29231906_776f_11ec_a656_0a43f95f28a3'" commerce +$ vtctldclient ApplySchema --ddl_strategy='vitess --postpone-completion --allow-concurrent' --sql "revert vitess_migration '29231906_776f_11ec_a656_0a43f95f28a3'" commerce 3cc4ae0e_776f_11ec_a656_0a43f95f28a3 ``` @@ -141,7 +151,7 @@ The above begins a `REVERT` migration that is open-ended (does not complete), vi Finally, if we are satisfied that the `drop index` migration went well, we issue: ```sh -$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' cancel" commerce +$ vtctldclient ApplySchema --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' cancel" commerce ``` That is, we cancel the `REVERT` operation. @@ -149,7 +159,7 @@ That is, we cancel the `REVERT` operation. Or, should we have not dropped the index? If our migration seems to have been wrong, we run: ```sh -$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' complete" commerce +$ vtctldclient ApplySchema --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' complete" commerce ``` Which means we want to apply the revert. Since the revert is already running in the background, it is likely that binary log processing is up to date, and cut-over is near instantaneous. @@ -171,7 +181,7 @@ Note that there can be scenarios with impossible ordering. Those hardly make sen An example for in-order submission: ```sh -$ vtctldclient ApplySchema --skip_preflight --ddl_strategy='vitess --allow-concurrent --in-order-completion' --sql "create table t1 (id int primary key); create view v1 as select id from t1;" commerce +$ vtctldclient ApplySchema --ddl_strategy='vitess --allow-concurrent --in-order-completion' --sql "create table t1 (id int primary key); create view v1 as select id from t1;" commerce ``` {{< info >}} diff --git a/content/en/docs/18.0/user-guides/schema-changes/audit-and-control.md b/content/en/docs/18.0/user-guides/schema-changes/audit-and-control.md index 118fcbee9..b2b3d5f9f 100644 --- a/content/en/docs/18.0/user-guides/schema-changes/audit-and-control.md +++ b/content/en/docs/18.0/user-guides/schema-changes/audit-and-control.md @@ -60,13 +60,13 @@ mysql> drop table customer; You may use `vtctl` or `vtctlclient` (the two are interchangeable for the purpose of this document) to apply schema changes. The `ApplySchema` command supports both synchronous and online schema migrations. To run an online schema migration you will supply the `--ddl_strategy` command line flag: ```shell -$ vtctlclient -- ApplySchema --ddl_strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED" commerce +$ vtctldclient ApplySchema --ddl-strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED" commerce a2994c92_f1d4_11ea_afa3_f875a4d24e90 ``` You my run multiple migrations withing the same `ApplySchema` command: ```shell -$ vtctlclient -- ApplySchema --skip_preflight --ddl_strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED; CREATE TABLE sample (id int PRIMARY KEY); DROP TABLE another;" commerce +$ vtctldclient ApplySchema --ddl-strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED; CREATE TABLE sample (id int PRIMARY KEY); DROP TABLE another;" commerce 3091ef2a_4b87_11ec_a827_0a43f95f28a3 ``` @@ -75,7 +75,6 @@ $ vtctlclient -- ApplySchema --skip_preflight --ddl_strategy "vitess" --sql "ALT - `--ddl_strategy`: by default migrations run directly via MySQL standard DDL. This flag must be aupplied to indicate an online strategy. See also [DDL strategies](../ddl-strategies) and [ddl_strategy flags](../ddl-strategy-flags). - `--migration_context] [--request_context= ] {--sql=<sql> || --sql-file=<filename>} <keyspace>