oceanbase-ce 3.1.2 于2021年12月31日发布,计划升级原有各组件版本,文中对下面几个问题进行思考并答复:
1) 请问所有组件如何升级(不限于obd、oceanbase-ce、obagent、obproxy、obclient)
2) 所有组件升级时,对生产业务有什么影响,是否建议安排停机时间,停止应用等?
3) 如何进行回退?

0. ENV


obd 1.1.1(升级到1.2.1)
oceanbase-ce 3.1.1(升级到3.1.2)
obproxy 3.1.0(升级到3.1.2)
obagent 1.0.0(升级到1.1.0)
obclient-2.0.0-2.el7.x86_64目前(2022-01-07)无新版本
当前采用单机3 observer + 3 obproxy方式部署。
1. OBD升级


各组件升级前,需要先将OBD升级到 V1.2.1。
1.1 OBD升级前查看版本--需要使用root用户进行升级
使用admin用户升级提示,需要以root用户执行
[admin@ob3 ~]$ obd update[ERROR] To update OBD, you must be a root user.
查看obd当前版本为1.1.1
[root@ob3 ~]# obd --versionOceanBase Deploy: 1.1.1REVISION: eca0509213255ef714d350a5af897b381f9244eaBUILD_BRANCH: masterBUILD_TIME: Sep 30 2021 08:54:49OURCECopyright (C) 2021 OceanBaseLicense GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.This is free software: you are free to change and redistribute it.There is NO WARRANTY, to the extent permitted by law.
1.2 OBD升级
OBD升级到1.2.1
[root@ob3 ~]# obd updateUpdate OceanBase-community-stable-el7 okUpdate OceanBase-development-kit-el7 okDownload ob-deploy-1.2.1-9.el7.x86_64.rpm (31.76 M): 100% [#########################################################################################################] Time: 0:00:06 4.77 MB/sFound a higher version package for OBDname: ob-deployversion: 1.2.1release:9.el7arch: x86_64md5: 219823e1119d37f59ce2c906b44e45605acfd3f3Upgrade successful.Current version : 1.2.1
1.3 OBD升级后查看版本
升级后版本为1.2.1
[root@ob3 ~]# obd --versionOceanBase Deploy: 1.2.1REVISION: 9f2fb49eca02d9a5efecaab5994fe7abddf81a79BUILD_BRANCH: (HEADBUILD_TIME: Jan 04 2022 13:23:42OURCECopyright (C) 2021 OceanBaseLicense GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.This is free software: you are free to change and redistribute it.There is NO WARRANTY, to the extent permitted by law.
2. oceanbase-ce升级


2.1 升级组件语法
升级一个已经启动的组件
obd cluster upgrade <deploy_name> -c <component_name> -V <version> [tags]
2.2 升级
将oceanbase-ce 3.1.1升级为oceanbase-ce 3.1.2。
[admin@ob3 ~]$ obd cluster upgrade rundba -c oceanbase-ce -V 3.1.2Get local repositories and plugins ok+----------------------------------------------------------------------------------------------------------------------------+| Search oceanbase-ce 3.1.2 Result |+--------------+---------+-----------------------+--------+------------------------------------------------------------------+| name | version | release | arch | md5 |+--------------+---------+-----------------------+--------+------------------------------------------------------------------+| oceanbase-ce | 3.1.2 | 10000392021123010.el7 | x86_64 | f38723204d49057d3e062ffad778edc1552a7c114622bf2a86fea769fbd202ea |+--------------+---------+-----------------------+--------+------------------------------------------------------------------+Download oceanbase-ce-3.1.2-10000392021123010.el7.x86_64.rpm (46.45 M): 100% [######################################################################################] Time: 0:00:12 3.91 MB/sinstall oceanbase-ce-3.1.2 for local okOpen ssh connection okConnect to observer ok+------------------------------------------------------------------------------------------------------------+| Packages Will Be Used |+--------------+---------+-----------------------+--------+------------------------------------------+-------+| name | version | release | arch | md5 | mark |+--------------+---------+-----------------------+--------+------------------------------------------+-------+| oceanbase-ce | 3.1.1 | 4.el7 | x86_64 | f19f8bfb67723712175fb0dfd60579196b3168f1 | start | #升级前版本为3.1.1| oceanbase-ce | 3.1.2 | 10000392021123010.el7 | x86_64 | 7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed | dest | #升级后版本为3.1.2+--------------+---------+-----------------------+--------+------------------------------------------+-------+If you use a non-official release, we cannot guarantee a successful upgrade or technical support when you fail. Make sure that you want to use the above package to upgrade. [y/n]: y #按提示输入y进行升级Remote oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed repository install okRemote oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed repository lib check !![WARN] ob1(127.0.0.1) oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed require: libmariadb.so.3[WARN] ob2(127.0.0.1) oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed require: libmariadb.so.3[WARN] ob3(127.0.0.1) oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed require: libmariadb.so.3Try to get lib-repositoryDownload oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm (155.22 K): 100% [################################################################################] Time: 0:00:00 3.90 MB/sFound a higher versionname: oceanbase-ce-libsversion: 3.1.2release:10000392021123010.el7arch: x86_64md5: 94fff0ab31de053051dba66039e3185fa390cad5Do you want to use it? [y/n]: y #按提示输入y,使用oceanbase-ce-libs 3.1.2Package oceanbase-ce-libs-3.1.2 is available.install oceanbase-ce-libs-3.1.2 for local okUse oceanbase-ce-libs-3.1.2-94fff0ab31de053051dba66039e3185fa390cad5 for oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aedRemote oceanbase-ce-libs-3.1.2-94fff0ab31de053051dba66039e3185fa390cad5 repository install okRemote oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed repository lib check okStart observer okobserver program health check okConnect to observer okExec upgrade_checker.py okEnable upgrade mode okExec upgrade_pre.py okRotation upgrade okupgrade zone "zone1"upgrade zone "zone2"upgrade zone "zone3"Exec upgrade_post.py okDisable upgrade mode okRoot inspection okExec upgrade_post_checker.py okWait for observer init ok+---------------------------------------------+| observer |+-----------+---------+------+-------+--------+| ip | version | port | zone | status |+-----------+---------+------+-------+--------+| 127.0.0.1 | 3.1.2 | 2881 | zone1 | active || 127.0.0.1 | 3.1.2 | 3881 | zone2 | active || 127.0.0.1 | 3.1.2 | 4881 | zone3 | active |+-----------+---------+------+-------+--------+
2.3 查看升级后集群信息
集群已升级到3.1.2
[admin@ob3 ~]$ obd cluster display rundbaGet local repositories and plugins okOpen ssh connection okCluster status check okConnect to observer okWait for observer init ok+---------------------------------------------+| observer |+-----------+---------+------+-------+--------+| ip | version | port | zone | status |+-----------+---------+------+-------+--------+| 127.0.0.1 | 3.1.2 | 2881 | zone1 | active || 127.0.0.1 | 3.1.2 | 3881 | zone2 | active || 127.0.0.1 | 3.1.2 | 4881 | zone3 | active |+-----------+---------+------+-------+--------+Connect to obproxy ok+---------------------------------------------+| obproxy |+-----------+------+-----------------+--------+| ip | port | prometheus_port | status |+-----------+------+-----------------+--------+| 127.0.0.1 | 2883 | 2888 | active || 127.0.0.1 | 3883 | 3888 | active || 127.0.0.1 | 4883 | 4888 | active |+-----------+------+-----------------+--------+
3. 升级obproxy


3.1 将rundba集群的obproxy升级到3.2.0
[admin@ob3 ~]$ obd cluster upgrade rundba -c obproxy -V 3.2.0Get local repositories and plugins ok+---------------------------------------------------------------------------------------------------------+| Search obproxy 3.2.0 Result |+---------+---------+---------+--------+------------------------------------------------------------------+| name | version | release | arch | md5 |+---------+---------+---------+--------+------------------------------------------------------------------+| obproxy | 3.2.0 | 1.el7 | x86_64 | c2ad988860a0fe5c07cbe3755fab0e2c165cf7a771e6c1f11741cc7208bf13d3 |+---------+---------+---------+--------+------------------------------------------------------------------+Download obproxy-3.2.0-1.el7.x86_64.rpm (7.80 M): 100% [############################################################################################################install obproxy-3.2.0 for local okOpen ssh connection ok[WARN] No such upgrade_route plugin for obproxy-3.1.0[WARN] No such upgrade_check plugin for obproxy-3.1.0[WARN] No such upgrade_check plugin for obproxy-3.2.0+-----------------------------------------------------------------------------------------+| Packages Will Be Used |+---------+---------+---------+--------+------------------------------------------+-------+| name | version | release | arch | md5 | mark |+---------+---------+---------+--------+------------------------------------------+-------+| obproxy | 3.1.0 | 1.el7 | x86_64 | 0b17cf0459a3b53c5a2febb6572894d183154c64 | start | #升级前版本为3.1.0| obproxy | 3.2.0 | 1.el7 | x86_64 | 8d5c6978f988935dc3da1dbec208914668dcf3b2 | dest | #升级前版本为3.2.0+---------+---------+---------+--------+------------------------------------------+-------+If you use a non-official release, we cannot guarantee a successful upgrade or technical support when you fail. Make sure that you want to use the above package to upgrade. [y/n]: y #按提示输入y进行升级Remote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository install okRemote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository lib check okStop obproxy okStart obproxy okobproxy program health check x[WARN] failed to start obproxy1(127.0.0.1) obproxy #升级obproxy成功,但启动obproxy1失败。
3.2 尝试重启obproxy--失败
[admin@ob3 ~]$ obd cluster restart rundba -c obproxy[ERROR] Deploy "rundba" is upgrading. You could not start an upgrading cluster.
3.3 再次运行升级--成功
[admin@ob3 ~]$ obd cluster upgrade rundba -c obproxy -V 3.2.0Get local repositories and plugins okOpen ssh connection okRemote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository install okRemote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository lib check okStop obproxy okStart obproxy okobproxy program health check okConnect to obproxy ok+---------------------------------------------+| obproxy |+-----------+------+-----------------+--------+| ip | port | prometheus_port | status |+-----------+------+-----------------+--------+| 127.0.0.1 | 2883 | 2888 | active || 127.0.0.1 | 3883 | 3888 | active || 127.0.0.1 | 4883 | 4888 | active |+-----------+------+-----------------+--------+
4. obagent升级


将rundba集群的obagent升级到1.1.0
[admin@ob3 ~]$ obd cluster upgrade obagent -c obagent -V 1.1.0Get local repositories and plugins ok+---------------------------------------------------------------------------------------------------------+| Search obagent 1.1.0 Result |+---------+---------+---------+--------+------------------------------------------------------------------+| name | version | release | arch | md5 |+---------+---------+---------+--------+------------------------------------------------------------------+| obagent | 1.1.0 | 1.el7 | x86_64 | ac990f4250748dc29a0842ca5093503037d049c38677a2543947b70d6797e7a5 |+---------+---------+---------+--------+------------------------------------------------------------------+Download obagent-1.1.0-1.el7.x86_64.rpm (7.74 M): 100% [#################################################################################] Time: 0:00:01 4.38 MB/sinstall obagent-1.1.0 for local okOpen ssh connection ok[WARN] No such upgrade_route plugin for obagent-1.0.0[WARN] No such upgrade_check plugin for obagent-1.0.0[WARN] No such upgrade_check plugin for obagent-1.1.0+-----------------------------------------------------------------------------------------+| Packages Will Be Used |+---------+---------+---------+--------+------------------------------------------+-------+| name | version | release | arch | md5 | mark |+---------+---------+---------+--------+------------------------------------------+-------+| obagent | 1.0.0 | 1.el7 | x86_64 | e45f0daf05a9995843c799014fb59d388d51eae2 | start || obagent | 1.1.0 | 1.el7 | x86_64 | d2416fadeadba35944872467843d55da0999f298 | dest |+---------+---------+---------+--------+------------------------------------------+-------+If you use a non-official release, we cannot guarantee a successful upgrade or technical support when you fail. Make sure that you want to use the above package to upgrade. [y/n]: y #按提示输入y进行升级Remote obagent-1.1.0-d2416fadeadba35944872467843d55da0999f298 repository install okRemote obagent-1.1.0-d2416fadeadba35944872467843d55da0999f298 repository lib check okStop obagent okStart obproxy okobagent program health check ok+-----------------------------------------------+| obagent |+-----------+-------------+------------+--------+| ip | server_port | pprof_port | status |+-----------+-------------+------------+--------+| 127.0.0.1 | 8088 | 8089 | active || 127.0.0.1 | 9088 | 9089 | active || 127.0.0.1 | 10088 | 10089 | active |+-----------+-------------+------------+--------+
5. obclient升级


本次obclient无新版本,不做升级。
升级方法:
obd cluster upgrade rundba -c obclient -V 2.0.0
或
[admin@ob3 ~]$ yum -y install obclient
6. 升级说明


OBD、obclient、obagent升级对业务无影响;
OceanBase在zone数大于等于3的情况下是轮转升级,Oceanbase-ce升级时,理论上业务上是无感知的;
obproxy的升级导致链接断开,需要重新建链,对正在运行的业务有影响;
对oceanbase-ce、obproxy的升级建议安排合理的停机窗口进行,目前社区版短期没有规划回退方案,也没法提供升级后的回滚能力,建议升级前做好备份和升级故障应急预案。
7. ref


https://github.com/oceanbase/obdeploy/blob/master/docs/docs-cn/obd-commands/cluster-commands.mdhttps://github.com/oceanbase/obdeploy/blob/master/docs/docs-cn/obd-commands/cluster-commands.mdhttps://github.com/oceanbase/obdeploy/blob/master/docs/docs-cn/Q%26A.mdhttps://github.com/oceanbase/oceanbase/releases
旨在交流,不足之处,还望抛砖。
作者:王坤,微信公众号:rundba,欢迎转载,转载请注明出处。
如需公众号转发,请联系wx: landnow。





