TIDB-BR备份工具

原创冰凝云和恩墨交付战队 2021-06-28

1200

备份文件在各个 tikv节点：

备份脚本：
备份目录的用户所属组是 tidb.tidb 。。备份脚本部署在 pd节点 61 , 61备份目录下存放的是备份的元数据； tikv 各节点的备份目录下放的是实际的备份数据；

备份路径下会生成以下两种类型文件：

SST 文件：存储 TiKV 备份下来的数据信息

backupmeta 文件：存储本次备份的元信息，包括备份文件数、备份文件的 Key 区间、备份文件大小和备份文件 Hash (sha256) 值

br备份工具最好用 v3.1.0 用v4.0.0 会报错：["update GC safePoint with TTL failed"]

cat bak_table.sh
/apps/tidb-toolkit-v3.1.0-linux-amd64/bin/br backup table \
--pd "10.128.3.61:2379" \
--db test \
--table rlog \
--storage "local:///apps/backup/table" \
--ratelimit 120 \
--log-file bak_rlog.log

[root@vpc-test3-tidb backup]# ./bak_table.sh
Detial BR log in bak_rlog.log
Table backup <-----------------------------------------------------------------------------------------------------> 100.00%
Checksum <---------------------------------------------------------------------------------------------------------> 100.00%
[2020/07/02 15:10:33.139 +08:00] [INFO] [collector.go:58] ["Table backup Success summary: total backup ranges: 2, total success: 2, total failed: 0, total take(s): 0.06, total kv: 92, total size(Byte): 8647, avg speed(Byte/s): 145086.75"] ["backup checksum"=9.656093ms] ["backup fast checksum"=723.065碌s] ["backup total regions"=2]
[root@vpc-test3-tidb backup]#

恢复 test_mmststdb.rlog表到 test数据库
cat restore_table.sh
/apps/tidb-toolkit-v3.1.0-linux-amd64/bin/br restore table \
--pd "10.128.3.61:2379" \
--db "test" \
--table "rlog" \
--storage "local:///apps/backup/table" \
--log-file restore_rlog.log

[root@vpc-test3-tidb backup]# ./restore_table.sh
Detial BR log in restore_rlog.log
Table restore <---------------------------------------------------------------------------------------------------|> 100.00%Error: Cannot read local:///apps/backup/table/4_2_890_e5f6dccc0d2320d5775077dbe42c7375302485dbedd3deefc29eb76ead8d6abb_write.sst: No such file or directory (os error 2): download sst failed

日志：
[2020/07/02 15:14:18.768 +08:00] [WARN] [backoff.go:74] ["unexcepted error, stop to retry"] [error="Cannot read local:///app
s/backup/table/4_2_890_7b01ac24e4f666fdf8d3796da5c123f8abfbee287431a2a82f9609eae4c56899_write.sst: No such file or directory
(os error 2): download sst failed"] [errorVerbose="download sst failed\nCannot read local:///apps/backup/table/4_2_890_7b01
[root@vpc-test3-tidb backup]#

备份一个数据库：

cat bak_db.sh
/apps/tidb-toolkit-v3.1.0-linux-amd64/bin/br backup db \
--pd "10.128.3.61:2379" \
--db test_mmststdb \
--storage "local:///apps/backup/db" \
--ratelimit 120 \
--log-file bak_test_mmststdb.log

Detial BR log in bak_test_mmststdb.log
Database backup <----................................................................................................> 3.09%
Database backup <-----\..............................................................................................> 5.25%
Database backup <-------|............................................................................................> 7.72%
Checksum <----------------------------------------------------------------------------------------------------------> 99.41%Checksum <---------------------------------------------------------------------------------------------------------\> 99.41%Checksum <---------------------------------------------------------------------------------------------------------> 100.00%[2020/07/02 15:32:30.008 +08:00] [INFO] [collector.go:58] ["Database backup Success summary: total backup ranges: 224, total success: 224, total failed: 0, total take(s): 46.70, total size(MB): 6847.14, avg speed(MB/s): 146.63, total kv: 83349128"] ["backup checksum"=3.775083303s] ["backup fast checksum"=61.025875ms] ["backup total regions"=324]

备份整个集群：

cat bak_full.sh

/apps/tidb-toolkit-v3.1.0-linux-amd64/bin/br backup full \

--pd "10.128.3.61:2379" \

--storage "local:///apps/backup/full" \

--ratelimit 120 \

--log-file bak_full_20200703.log

增量备份的话 tikv_gc_life_time 参数必须设置大于 7天，，最后设置成 720小时。

SELECT * FROM mysql.tidb WHERE VARIABLE_NAME = 'tikv_gc_life_time';
UPDATE mysql.tidb SET VARIABLE_VALUE = '240h' WHERE VARIABLE_NAME = 'tikv_gc_life_time';

br 定时备份任务脚本

30 0 * * 3,7 /home/tidb/br_full.sh > /home/tidb/br_full.log

30 0 * * 1,2 /home/tidb/br_incr.sh > /home/tidb/br_incr.log

30 0 * * 4,5,6 /home/tidb/br_incr2.sh > /home/tidb/br_incr2.log

[tidb@vpc-test3-tidb ~]$
[tidb@vpc-test3-tidb ~]$ cat br_full.sh
#!/bin/bash
dt=` date +%Y%m%d`
ydt=` date '-d 7 day ago' +%Y%m%d`

cd /apps/backup/
mkdir /apps/backup/full_$dt

/apps/tidb-toolkit-v3.1.0-linux-amd64/bin/br backup full \
--pd "10.128.3.61:2379" \
--storage "local:///apps/backup/full_$dt" \
--ratelimit 120 \
--log-file bak_full_$dt.log

rm -rf /apps/backup/full_$ydt

[tidb@vpc-test3-tidb ~]$

增量备份1：

[tidb@vpc-test3-tidb ~]$ cat br_incr.sh

#!/bin/bash

dt=` date +%Y%m%d`

ydt=` date '-d 7 day ago' +%Y%m%d`

last_br_full=`date -d "last Sunday" +%Y%m%d`

cd /apps/backup/

mkdir /apps/backup/incr_$dt

dir=full_$last_br_full

echo $dir

LAST_BACKUP_TS=`/apps/tidb-toolkit-v3.1.0-linux-amd64/bin/br validate decode --field="end-version" -s local:///apps/backup/$dir`

echo $LAST_BACKUP_TS

/apps/tidb-toolkit-v3.1.0-linux-amd64/bin/br backup full\

--pd "10.128.3.61:2379"\

-s local:///apps/backup/incr_$dt \

--lastbackupts ${LAST_BACKUP_TS} \

--log-file bak_incr_$dt.log

rm -rf /apps/backup/incr_$ydt

增量备份2：

[tidb@vpc-test3-tidb ~]$

[tidb@vpc-test3-tidb ~]$ cat br_incr2.sh