暂无图片
暂无图片
暂无图片
暂无图片
暂无图片

GBase RTSync同步工具高可用功能实战

wj2021 2021-05-22
826

GBase RTSync是南大通用的一款异构及同构数据库增量数据实时同步产品。为了保证数据同步的可用性,可以配置RTSync高可用模式。RTSync高可用依赖于Zookeeper服务。配置多套RTSync组件后,通过Zookeeper服务选举主节点后对外提供服务。当主节点服务异常退出后,通过Zookeeper服务会重新选举其他节点作为主来达到高可用的目的。

下面模拟一个项目来说明如何部署高可用服务

项目背景

项目需要将Oracle 11g中的增量数据同步到GBase 8s中,且该项目需要支持高可用服务。

部署环境

RTsync服务本身依赖于Zookeeper和kafka,为了保证Zookeeper和Kafka的高可用性,故准备了三台服务器,其ip为192.168.2.201,192.168.2.202,192.168.2.203。这三台机器用于部署三节点的zookeeper和kafka集群。

同时准备两台机器,其ip为192.168.2.204,192.168.2.205。这两台机器用于部署两套RTSync,用作高可用使用。

环境及工具安装

安装Zookeeper,将Zookeeper安装到相应的三台服务器上,可参考网上的教程;

安装Kafka,将Kafka安装到相应的三台服务器上,可参考网上的教程;

RTSync组件安装:

将RTSync分别复制到到204的/opt/目录下,目录为/opt/RTSync;205的/opt/目录下,目录为/opt/RTSync。

工具配置

首先配置RTSync使用的Zookeeper和Kafka,按照手册中的描述,修改config_kafka_ora.properties文件中的五个属性(其他属性采用默认即可):
bootstrap.servers=192.168. 2.201 :9092,192.168. 2.20 2:9092,192.168. 2.20 3:9092

zookeeper.connect= 192.168. 2.201 :2181,192.168. 2.20 2:2181,192.168. 2.20 3:2181

topic.name=test01

group.id=testgid

topic.replication.num=3

以上配置需要在204,205上同时修改。

然后配置RTSync的配置文件config_task.xml
针对204节点机器,修改配置为如下:

 <manager   ip = "192.168.2.20 4 "   port = "9432"   heartbeatPort = "9000"   httpPort = "8080"   isTableHotPatch = "true" />  

 <source   ip = "192.168.2.20 4 "   path = "/opt/RTSync"   readParseAdapter = "adapter"   user = "root"   password = " ******** "   queueSize = "10000"   openMonitor = "true"   monitorInterval = "300"   rpcPort = "9191"   sshPort = "22"   dbObjToUpperCase = "false"   isConvertSingleQuote = "true"   queuePollTimeOut = "600"   isEmptyStrPkEqualsNull = "true"   isAllowInsertPkNull = "true" />

 <target   ip = "192.168.2.20 4 "   path = "/opt/RTSync"   writeDataAdapter = "adapter"   user = "root"   password = " ******** "   errorishandle = " false "   sendDataBySocket = " false "   rpcPort = "9191"   sshPort = "22" />  

 <mappings>  

     <source-target   id = "sync1" >  

     <db>  

         <sourcedb  

             charset = "UTF8"  

             type = "ORACLE"  

             startLSN = "0"  

             fetchSize = "500"  

             oracleScnStep = "50000"  

             timestampWithFraction = "false"  

             maxRecordsPerRead = "200"  

             maxSizeOfPerRecord = "1024"  

             timeOut = "2"  

             driver = "oracle.jdbc.OracleDriver"  

             url = "jdbc:oracle:thin@192.168.2.108:1521/orcl"  

             user = " test "  

             password = " ******** "  

             catalog = " TEST "

             parallel = "3"   >  

         </sourcedb>

         <targetdb  

             charset = "UTF8"  

             type = " GBASE8S "  

             commitSize = "1000"  

             queueSize = "2000"  

             user = " gbasedbt "  

             password = " ******** "  

             driver = "com. gbasedbt .jdbc.Driver"  

             catalog = "test"  

             timeOut = "2"  

               parrallel = " 3 "

             url = "jdbc:gbase dbt-sqli ://192.168.2.105: 3000/sysmaster:gbasedbtserver=gbasetest " >  

         </targetdb>

         <tables   isInclude = "true" >

             <table   deleteMode = "NORMAL"   sourceTableName = "T1"   sourcePkColName = " COL1 "   targetTableName = "t1"   targetPkColName = " col1 "   />

         </tables>  

     </db>  

     </source-target>  

 </mappings>  

针对205节点机器,修改配置为如下:

 <manager   ip = "192.168.2.20 5 "   port = "9432"   heartbeatPort = "9000"   httpPort = "8080"   isTableHotPatch = "true" />  

 <source   ip = "192.168.2.20 5 "   path = "/opt/RTSync"   readParseAdapter = "adapter"   user = "root"   password = " ******** "   queueSize = "10000"   openMonitor = "true"   monitorInterval = "300"   rpcPort = "9191"   sshPort = "22"   dbObjToUpperCase = "false"   isConvertSingleQuote = "true"   queuePollTimeOut = "600"   isEmptyStrPkEqualsNull = "true"   isAllowInsertPkNull = "true" />

 <target   ip = "192.168.2.20 5 "   path = "/opt/RTSync"   writeDataAdapter = "adapter"   user = "root"   password = " ******** "   errorishandle = " false "   sendDataBySocket = " false "   rpcPort = "9191"   sshPort = "22" />  

 <mappings>  

     <source-target   id = "sync1" >  

     <db>  

         <sourcedb  

             charset = "UTF8"  

             type = "ORACLE"  

             startLSN = "0"  

             fetchSize = "500"  

             oracleScnStep = "50000"  

             timestampWithFraction = "false"  

             maxRecordsPerRead = "200"  

             maxSizeOfPerRecord = "1024"  

             timeOut = "2"  

             driver = "oracle.jdbc.OracleDriver"  

             url = "jdbc:oracle:thin@192.168.2.108:1521/orcl"  

             user = " test "  

             password = " ******** "  

             catalog = " TEST "

             parallel = "3"   >  

         </sourcedb>

         <targetdb  

             charset = "UTF8"  

             type = " GBASE8S "  

             commitSize = "1000"  

             queueSize = "2000"  

             user = " gbasedbt "  

             password = " ******** "  

             driver = "com. gbasedbt .jdbc.Driver"  

             catalog = "test"  

             timeOut = "2"  

               parrallel = " 3 "

             url = "jdbc:gbase dbt-sqli ://192.168.2.105: 3000/sysmaster:gbasedbtserver=gbasetest " >  

         </targetdb>

         <tables   isInclude = "true" >

             <table   deleteMode = "NORMAL"   sourceTableName = "T1"   sourcePkColName = " COL1 "   targetTableName = "t1"   targetPkColName = " col1 "   />

         </tables>  

     </db>  

     </source-target>  

 </mappings>  

注:为了进行测试,oracle建了个测试用户test,在其中建了张表T1,这里要注意的是Oracle中建表以后默认是大写表名,在这个配置文件中也要写成大写。

启动同步

启动Zookeeper和Kafka;

分别启动204及205节点上的RTSync,执行sh /opt/RTSync/RTSyncManagerServer.sh start命令;

因为优先启动的204节点,所以默认启动后204节点作为主对外提供服务。

测试同步

1)正常同步测试

同步工具已经启动好了,在oracle中插入一条数据,测试一下:

i nsert into t1 values( 1, ‘asdf’);

commit;

然后在GBase 8s执行查询语句:

select * from test: t1;

结果显示:

col1 1

col2 asdf

1 row(s) retrieved

好了,数据已经同步过来了。

2) 高可用切换测试

默认主节点是204节点(优先启动的是主节点),在204节点/opt/RTSync目录下执行./RTSyncManagerServer.sh stop,把当前主节点服务停掉。

然后在oracle中插入一条数据,测试一下:

i nsert into t1 values( 2 , ‘ def ’);

commit;

然后在GBase 8s执行查询语句:

select * from t1;

结果显示:

col1 1

col2 asdf

col1 2

col2 def

2 row(s) retrieved

好了,数据已经同步过来了。

总结

总体来说,高可用搭建比较简单,参考RTSync使用手册即可。需要注意下高可用开启的条件是修改config_task.xml的server节点属性,修改为如下即可

isHighAvailable=" true "

dataRecoveryMode=" zookeeper "

————————————————
版权声明:本文为CSDN博主「qq_39280087」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。
原文链接:https://blog.csdn.net/qq_39280087/article/details/117152206

最后修改时间:2021-05-22 10:36:08
「喜欢这篇文章,您的关注和赞赏是给作者最好的鼓励」
关注作者
【版权声明】本文为墨天轮用户原创内容,转载时必须标注文章的来源(墨天轮),文章链接,文章作者等基本信息,否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容,欢迎发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

评论