+
+
+
+You can use Docker to quickly deploy a test MySQL 8.0 instance.
+
+1. Run a MySQL 8.0 Docker container:
```shell
- tiup dm deploy dm-test 8.5.0 topology.yaml -p
+ docker run --name mysql80 \
+ -e MYSQL_ROOT_PASSWORD=MyPassw0rd! \
+ -p 3306:3306 \
+ -d mysql:8.0
```
-## Step 2: Prepare the data source
+2. Connect to MySQL:
-You can use one or multiple MySQL instances as an upstream data source.
+ ```shell
+ docker exec -it mysql80 mysql -uroot -pMyPassw0rd!
+ ```
-1. Create a configuration file for each data source as follows:
+3. Create a dedicated user with required privileges for DM testing:
- {{< copyable "shell-regular" >}}
+ ```sql
+ CREATE USER 'tidb-dm'@'%'
+ IDENTIFIED WITH mysql_native_password
+ BY 'MyPassw0rd!';
- ```yaml
- source-id: "mysql-01"
- from:
- host: "127.0.0.1"
- user: "root"
- password: "fCxfQ9XKCezSzuCD0Wf5dUD+LsKegSg="
- port: 3306
+ GRANT PROCESS, BACKUP_ADMIN, RELOAD, REPLICATION SLAVE, REPLICATION CLIENT, SELECT ON *.* TO 'tidb-dm'@'%';
```
-2. Add the source to the DM cluster by running the following command. `mysql-01.yaml` is the configuration file created in the previous step.
+4. Create sample data:
- {{< copyable "shell-regular" >}}
+ ```sql
+ CREATE DATABASE hello;
+ USE hello;
- ```bash
- tiup dmctl --master-addr=127.0.0.1:8261 operate-source create mysql-01.yaml # use one of master_servers as the argument of --master-addr
+ CREATE TABLE hello_tidb (
+ id INT AUTO_INCREMENT PRIMARY KEY,
+ name VARCHAR(50)
+ );
+
+ INSERT INTO hello_tidb (name) VALUES ('Hello World');
+
+ SELECT * FROM hello_tidb;
```
-If you do not have a MySQL instance for testing, you can create a MySQL instance in Docker by taking the following steps:
+
+
+
-1. Create a MySQL configuration file:
+On macOS, you can quickly install and start MySQL 8.0 locally using [Homebrew](https://brew.sh).
- {{< copyable "shell-regular" >}}
+1. Update Homebrew and install MySQL 8.0:
```shell
- mkdir -p /tmp/mysqltest && cd /tmp/mysqltest
+ brew update
+ brew install mysql@8.0
+ ```
+
+2. Make MySQL commands accessible in the system path:
- cat > my.cnf <}}
+ ```shell
+ brew services start mysql@8.0
+ ```
+
+4. Connect to MySQL as the `root` user:
```shell
- docker run --name mysql-01 -v /tmp/mysqltest:/etc/mysql/conf.d -e MYSQL_ROOT_PASSWORD=my-secret-pw -d -p 3306:3306 mysql:5.7
+ mysql -uroot
```
-3. After the MySQL instance is started, access the instance:
+5. Create a dedicated user with required privileges for DM testing:
- > **Note:**
- >
- > This command is only suitable for trying out data migration, and cannot be used in production environments or stress tests.
+ ```sql
+ CREATE USER 'tidb-dm'@'%'
+ IDENTIFIED WITH mysql_native_password
+ BY 'MyPassw0rd!';
+
+ GRANT PROCESS, BACKUP_ADMIN, RELOAD, REPLICATION SLAVE, REPLICATION CLIENT, SELECT ON *.* TO 'tidb-dm'@'%';
+ ```
+
+6. Create sample data:
+
+ ```sql
+ CREATE DATABASE hello;
+ USE hello;
+
+ CREATE TABLE hello_tidb (
+ id INT AUTO_INCREMENT PRIMARY KEY,
+ name VARCHAR(50)
+ );
+
+ INSERT INTO hello_tidb (name) VALUES ('Hello World');
+
+ SELECT * FROM hello_tidb;
+ ```
+
+
+
+
+
+On Enterprise Linux distributions like CentOS, you can install MySQL 8.0 from the MySQL Yum repository.
- {{< copyable "shell-regular" >}}
+1. Download and install the MySQL Yum repository package from [MySQL Yum repository download page](https://dev.mysql.com/downloads/repo/yum). For Linux versions other than 9, you must replace the `el9` (Enterprise Linux version 9) in the following URL while keeping `mysql80` for MySQL version 8.0:
```shell
- mysql -uroot -p -h 127.0.0.1 -P 3306
+ sudo yum install -y https://dev.mysql.com/get/mysql80-community-release-el9-1.noarch.rpm
```
-## Step 3: Prepare a downstream database
+2. Install MySQL:
-You can choose an existing TiDB cluster as a target for data migration.
+ ```shell
+ sudo yum install -y mysql-community-server --nogpgcheck
+ ```
+
+3. Start MySQL:
+
+ ```shell
+ sudo systemctl start mysqld
+ ```
-If you do not have a TiDB cluster for testing, you can quickly build a demonstration environment by running the following command:
+4. Find the temporary root password in the MySQL log:
-{{< copyable "shell-regular" >}}
+ ```shell
+ sudo grep 'temporary password' /var/log/mysqld.log
+ ```
-```shell
-tiup playground
-```
+5. Connect to MySQL as the `root` user with the temporary password:
-## Step 4: Prepare test data
+ ```shell
+ mysql -uroot -p
+ ```
-Create a test table and data in one or multiple data sources. If you use an existing MySQL database, and the database contains available data, you can skip this step.
+6. Reset the `root` password:
-{{< copyable "sql" >}}
+ ```sql
+ ALTER USER 'root'@'localhost'
+ IDENTIFIED BY 'MyPassw0rd!';
+ ```
-```sql
-drop database if exists `testdm`;
-create database `testdm`;
-use `testdm`;
-create table t1 (id bigint, uid int, name varchar(80), info varchar(100), primary key (`id`), unique key(`uid`)) DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
-create table t2 (id bigint, uid int, name varchar(80), info varchar(100), primary key (`id`), unique key(`uid`)) DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
-insert into t1 (id, uid, name) values (1, 10001, 'Gabriel García Márquez'), (2, 10002, 'Cien años de soledad');
-insert into t2 (id, uid, name) values (3, 20001, 'José Arcadio Buendía'), (4, 20002, 'Úrsula Iguarán'), (5, 20003, 'José Arcadio');
-```
+7. Create a dedicated user with required privileges for DM testing:
-## Step 5: Create a data migration task
+ ```sql
+ CREATE USER 'tidb-dm'@'%'
+ IDENTIFIED WITH mysql_native_password
+ BY 'MyPassw0rd!';
-1. Create a task configuration file `testdm-task.yaml`:
+ GRANT PROCESS, BACKUP_ADMIN, RELOAD, REPLICATION SLAVE, REPLICATION CLIENT, SELECT ON *.* TO 'tidb-dm'@'%';
+ ```
- {{< copyable "" >}}
+8. Create sample data:
+
+ ```sql
+ CREATE DATABASE hello;
+ USE hello;
+
+ CREATE TABLE hello_tidb (
+ id INT AUTO_INCREMENT PRIMARY KEY,
+ name VARCHAR(50)
+ );
+
+ INSERT INTO hello_tidb (name) VALUES ('Hello World');
+
+ SELECT * FROM hello_tidb;
+ ```
+
+
+
+
+
+On Ubuntu, you can install MySQL from the official Ubuntu repository.
+
+1. Update your package list:
+
+ ```shell
+ sudo apt-get update
+ ```
+
+2. Install MySQL:
+
+ ```shell
+ sudo apt-get install -y mysql-server
+ ```
+
+3. Check whether the `mysql` service is running, and start the service if necessary:
+
+ ```shell
+ sudo systemctl status mysql
+ sudo systemctl start mysql
+ ```
+
+4. Connect to MySQL as the `root` user using socket authentication:
+
+ ```shell
+ sudo mysql
+ ```
+
+5. Create a dedicated user with required privileges for DM testing:
+
+ ```sql
+ CREATE USER 'tidb-dm'@'%'
+ IDENTIFIED WITH mysql_native_password
+ BY 'MyPassw0rd!';
+
+ GRANT PROCESS, BACKUP_ADMIN, RELOAD, REPLICATION SLAVE, REPLICATION CLIENT, SELECT ON *.* TO 'tidb-dm'@'%';
+ ```
+
+6. Create sample data:
+
+ ```sql
+ CREATE DATABASE hello;
+ USE hello;
+
+ CREATE TABLE hello_tidb (
+ id INT AUTO_INCREMENT PRIMARY KEY,
+ name VARCHAR(50)
+ );
+
+ INSERT INTO hello_tidb (name) VALUES ('Hello World');
+
+ SELECT * FROM hello_tidb;
+ ```
+
+
+
+
+
+## Step 3: Configure a TiDB DM source
+
+After preparing the source MySQL database, configure TiDB DM to connect to it. To do this, create a source configuration file with the connection details and apply the configuration using the `dmctl` tool.
+
+1. Create a source configuration file `mysql-01.yaml`:
+
+ > **Note:**
+ >
+ > This step assumes you have already created the `tidb-dm` user with replication privileges in the source database, as described in [Step 2](#step-2-prepare-a-source-database-optional).
+
+ ```yaml
+ source-id: "mysql-01"
+ from:
+ host: "127.0.0.1"
+ user: "tidb-dm"
+ password: "MyPassw0rd!" # In production environments, it is recommended to use a password encrypted with dmctl.
+ port: 3306
+ ```
+
+2. Create a DM data source:
+
+ ```shell
+ tiup dmctl --master-addr 127.0.0.1:8261 operate-source create mysql-01.yaml
+ ```
+
+## Step 4: Create a TiDB DM task
+
+After configuring the source database, you can create a migration task in TiDB DM. This task references the source MySQL instance and defines the connection details for the target TiDB database.
+
+1. Create a DM task configuration file `tiup-playground-task.yaml`:
```yaml
- name: testdm
- task-mode: all
+ # Task
+ name: tiup-playground-task
+ task-mode: "all" # Execute all phases - full data migration and incremental sync.
+ # Source (MySQL)
+ mysql-instances:
+ - source-id: "mysql-01"
+
+ ## Target (TiDB)
target-database:
host: "127.0.0.1"
port: 4000
user: "root"
- password: "" # If the password is not empty, it is recommended to use a password encrypted with dmctl.
+ password: "" # If the password is not empty, it is recommended to use a password encrypted with dmctl.
+ ```
- # Configure the information of one or multiple data sources
- mysql-instances:
- - source-id: "mysql-01"
- block-allow-list: "ba-rule1"
+2. Start the task using the configuration file:
+
+ ```shell
+ tiup dmctl --master-addr 127.0.0.1:8261 start-task tiup-playground-task.yaml
+ ```
+
+## Step 5: Verify the data replication
+
+After starting the migration task, verify whether data replication is working as expected. Use the `dmctl` tool to check the task status, and connect to the target TiDB database to confirm that the data has been successfully replicated from the source MySQL database.
- block-allow-list:
- ba-rule1:
- do-dbs: ["testdm"]
+1. Check the status of the TiDB DM task:
+
+ ```shell
+ tiup dmctl --master-addr 127.0.0.1:8261 query-status
```
-2. Create the task using dmctl:
+2. Connect to the target TiDB database:
+
+ ```shell
+ mysql --host 127.0.0.1 --port 4000 -u root --prompt 'tidb> '
+ ```
- {{< copyable "shell-regular" >}}
+3. Verify the replicated data. If you have created the sample data in [Step 2](#step-2-prepare-a-source-database-optional), you will see the `hello_tidb` table replicated from the MySQL source database to the target TiDB database:
- ```bash
- tiup dmctl --master-addr 127.0.0.1:8261 start-task testdm-task.yaml
+ ```sql
+ SELECT * FROM hello.hello_tidb;
```
-You have successfully created a task that migrates data from a `mysql-01` database to TiDB.
+ The output is as follows:
+
+ ```sql
+ +----+-------------+
+ | id | name |
+ +----+-------------+
+ | 1 | Hello World |
+ +----+-------------+
+ 1 row in set (0.00 sec)
+ ```
+
+## Step 6: Clean up (optional)
+
+After completing your testing, you can clean up the environment by stopping the TiUP Playground, removing the source MySQL instance (if created for testing), and deleting unnecessary files.
+
+1. Stop the TiUP Playground:
+
+ In the terminal where the TiUP Playground is running, press
+
+
+
+ To stop and remove the Docker container:
+
+ ```shell
+ docker stop mysql80
+ docker rm mysql80
+ ```
+
+
+
+
+
+ If you installed MySQL 8.0 using Homebrew solely for testing, stop the service and uninstall it:
+
+ ```shell
+ brew services stop mysql@8.0
+ brew uninstall mysql@8.0
+ ```
+
+ > **Note:**
+ >
+ > If you want to remove all MySQL data files, delete the MySQL data directory (commonly located at `/opt/homebrew/var/mysql`).
+
+
+
+
+
+ If you installed MySQL 8.0 from the MySQL Yum repository solely for testing, stop the service and uninstall it:
+
+ ```shell
+ sudo systemctl stop mysqld
+ sudo yum remove -y mysql-community-server
+ ```
+
+ > **Note:**
+ >
+ > If you want to remove all MySQL data files, delete the MySQL data directory (commonly located at `/var/lib/mysql`).
+
+
+
+
+
+ If you installed MySQL from the official Ubuntu repository solely for testing, stop the service and uninstall it:
+
+ ```shell
+ sudo systemctl stop mysql
+ sudo apt-get remove --purge -y mysql-server
+ sudo apt-get autoremove -y
+ ```
+
+ > **Note:**
+ >
+ > If you want to remove all MySQL data files, delete the MySQL data directory (commonly located at `/var/lib/mysql`).
+
+
+
+
+
+3. Remove the TiDB DM configuration files if they are no longer needed:
+
+ ```shell
+ rm mysql-01.yaml tiup-playground-task.yaml
+ ```
+
+4. If you no longer need TiUP, you can uninstall it:
+
+ ```shell
+ rm -rf ~/.tiup
+ ```
-After the task is created, you can use the `dmctl query-status` command to check the status of the task:
+## What's next
-{{< copyable "shell-regular" >}}
+Now that you successfully created a task that migrates data from a source MySQL database to a target TiDB database in a testing environment, you can:
-```bash
-tiup dmctl --master-addr 127.0.0.1:8261 query-status testdm
-```
+- Explore [TiDB DM Features](/dm/dm-overview.md)
+- Learn about [TiDB DM Architecture](/dm/dm-arch.md)
+- Set up [TiDB DM for a Proof of Concept or Production](/dm/deploy-a-dm-cluster-using-tiup.md)
+- Configure advanced [DM Tasks](/dm/dm-task-configuration-guide.md)