暂无图片
暂无图片
暂无图片
暂无图片
暂无图片

初探DM迁移MySQL到TIDB

IT那活儿 2021-04-22
1149
一、DM部署简介

DM部署方式有DM-Ansible、Binary和TIUP方式,TIUP为最新的部署方式,更加便捷。

以下步骤为使用DM-Ansible方式部署过程。

1. 安装依赖包


yum -y install epel-release git curl sshpasswget https://bootstrap.pypa.io/pip/2.7/get-pip.pypython get-pip.pywget https://download.pingcap.org/dm-ansible-v2.0.0.tar.gz[tidb@luyy01 ~]$ tar -zxvf dm-ansible-v2.0.0.tar.gz[tidb@luyy01 ~]$ mv dm-ansible-v2.0.0 dm-ansible[tidb@luyy01 ~]$sudo pip install -r ./requirements.txt

2. 在中控机上配置 SSH 互信和 sudo 规则


cd /home/tidb/dm-ansiblevi hosts.ini[servers]192.168.199.62192.168.199.63192.168.199.64[all:vars]username = tidb[tidb@luyy01 dm-ansible]$ ansible-playbook -i hosts.ini create_users.yml -u root -kSSH password:

3. 下载 DM 及监控组件安装包至中控机

ansible-playbook local_prepare.yml

4. 编辑 inventory.ini 配置文件


## DM modules[dm_master_servers]dm_master ansible_host=192.168.199.61 dm_master_port=18261[dm_worker_servers]dm-worker1 ansible_host=192.168.199.62 dm_worker_port=18262 source_id="mysql-replica-01"  mysql_host=192.168.199.61 mysql_user=admin mysql_password='YUZP1pa88fWbIzudw3bDTEBF6ZUTnw=='  mysql_port=3306[dm_portal_servers]dm_portal ansible_host=192.168.199.61## Monitoring modules[prometheus_servers]prometheus ansible_host=192.168.199.63[grafana_servers]grafana ansible_host=192.168.199.63[alertmanager_servers]alertmanager ansible_host=192.168.199.63## Global variables[all:vars]cluster_name = test-clusteransible_user = tidbdm_version = v2.0.0deploy_dir = home/tidb/deploygrafana_admin_user = "admin"grafana_admin_password = "admin"

5. 使用 dmctl 加密上游 MySQL 用户密码


cd /home/tidb/dm-ansible/resources/bin &&./dmctl -encrypt 'abc!@#123

6.修改内核参数并部署 DM 集群组件和监控组件

ansible-playbook deploy.yml

7. 启动DM集群

ansible-playbook start.yml
二、DM迁移数据
  

实例

主机

用户名

密码

端口

上游 MySQL

192.***.***.**

admin

admin

3306

下游 TiDB

192.***.***.**

root

4000

1. dm-master 进程配置


cd /home/tidb/deploy/confvi dm-master.tomlenable-gtid = false[[deploy]]source-id = "mysql-replica-01"dm-worker = "192.168.199.62:18262"

2. 配置任务

在dm-master上使用dm-portal生成task配置文件

登录网页http://192.***.***.**:8280/

可手动创建task文件


name: luyayaotask-mode: allis-sharding: falseclean-dump-file: trueignore-checking-items: [schema_of_shard_tables]target-database:  host: 192.168.199.63  port: 4000  user: root  password: KyMi25sGljNJPVL8OV3JrEA=mysql-instances:- source-id: mysql-replica-01  meta:    binlog-name: mysql-bin.0000001    binlog-pos: 4  filter-rules: []  route-rules:  - mysql-replica-01.route_rules.1  - mysql-replica-01.route_rules.2  black-white-list: mysql-replica-01.bw_list.1  mydumper-config-name: mysql-replica-01.dumproutes:  mysql-replica-01.route_rules.1:    schema-pattern: employees    table-pattern: ""    target-schema: employees    target-table: ""  mysql-replica-01.route_rules.2:    schema-pattern: employees    table-pattern: "*"    target-schema: employeesfilters: {}black-white-list:  mysql-replica-01.bw_list.1:    do-tables:    - db-name: employees      tbl-name: "*"    do-dbs: []    ignore-tables: []    ignore-dbs: []mydumpers:  mysql-replica-01.dump:    mydumper-path: bin/mydumper    threads: 4    chunk-filesize: 64    skip-tz-utc: true    extra-args: "--no-locks"

3. 进入交互窗口


./dmctl -master-addr 192.168.199.61:18261operate-source create home/tidb/deploy/conf/dm-master.yaml


cat dm-master.yamlserver-id: 101source-id: mysql-replica-01flavor: mysqlenable-relay: falseenable-gtid: falsefrom:  host: 192.168.199.61  user: admin  password: YUZP1pa88fWbIzudw3bDTEBF6ZUTnw==  port: 3306

4. 开始任务


start-task -s mysql-replica-01 /home/tidb/deploy/conf/luyayao.yaml» query-status   #查询任务运行状态

5. 日志查看

ps -ef|grep dm 可查看进程中dm-master 和 dm-work中的日志

通过查看日志和登录TIDB数据库发现数据已从DM完整迁移了过来。

END


多精彩干货分享

击下方名片关注

IT那活儿


文章转载自IT那活儿,如果涉嫌侵权,请发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

评论