【磐维数据库】binlog 缺失导致无法增量迁移

环境信息

• 源库：MySQL 5.7
• 目标库：磐维 2.0.2
• dtp：2.0.2

问题描述

• 使用 dtp 进行增量迁移，数据没同步过去；
• incremental.log 报错：

[ERROR][2024-06-13 11:36:44][cn.com.panwei.incremental.util.u.a(MysqlUtil.java:93)]查询mysql binlog初始状态异常:Access denied; you need (at least one of) the SUPER, REPLICATION CLIENT privilege(s) for this operation

[INFO ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.gr(MysqlBinlogReader.java:1555)]collector send finish record to queue for reconnect!
[WARN ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.gr(MysqlBinlogReader.java:1561)]mysql采集端开始重连, task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1, binlogfile:mysql-bin.000003, position:136882081
[WARN ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.gn(MysqlBinlogReader.java:441)]startRunListen 发生异常:sleep interrupted
[INFO ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.close(MysqlBinlogReader.java:190)]采集任务#taskID:【mysql_panwe12_ums_gogs#incremental#collect#incremental#1】, 【[采集端]子线程1[MySqlBinlog数据采集器]】关闭成功
[INFO ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.start(MysqlBinlogReader.java:139)]采集任务#taskID:【mysql_panwe12_ums_gogs#incremental#collect#incremental#1】, 数据采集器线程【blc-10.27.47.9:3306】启动成功, 开始从MySqlBinlog采集数据
[INFO ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.gr(MysqlBinlogReader.java:1570)]mysql采集端重连成功,task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1
[WARN ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.f.onDisconnect(MysqlBinlogReader.java:1631)]task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1 Stopped reading binlog, curBinlogFile:mysql-bin.000003, curBinlogPosition:136882081, currBeginPosition:136882081, lastAddRecordPosition:null
[INFO ][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.d.gm(MysqlBinlogReader.java:412)]起始点: mysql-bin.000003:136882081
[INFO ][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.d.gm(MysqlBinlogReader.java:417)]GTID: 740e9574-125d-11ee-bec3-6c92bf3bb513:1-1097101,edda3576-f616-11ed-a9b2-005056bc3dd9:1-2812960
[INFO ][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.d.gm(MysqlBinlogReader.java:422)]采集任务#taskID:【mysql_panwe12_ums_gogs#incremental#collect#incremental#1】, 正在尝试建立binlog读取器连接【30000】后超时失败
[INFO ][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.f.onConnect(MysqlBinlogReader.java:1598)]task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1 Connected to MySQL binlog at 10.27.47.9:3306, starting at mysql-bin.000003:136882081
[ERROR][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.f.onCommunicationFailure(MysqlBinlogReader.java:1603)]task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1 A communication failure event arrived
com.github.shyiko.mysql.binlog.network.ServerException: Could not open log file
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:937)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:606)
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:850)
	at java.lang.Thread.run(Thread.java:748)

• 关键信息为倒数第五行的：

com.github.shyiko.mysql.binlog.network.ServerException: Could not open log file

解决过程

• 开始怀疑是权限问题，重新执行增量前置步骤后，问题仍然存在；
• 实际原因是，源库有一个自动清理 binlog 的脚本，停止该定时任务后，不再报这个错误。