暂无图片
暂无图片
暂无图片
暂无图片
暂无图片
首页 / 分析obd日志了解其部署集群过程

分析obd日志了解其部署集群过程

原创 bicewow 2021-12-23
929

obd工具介绍

OceanBase Deployer（简称 OBD）是 OceanBase 开源软件的安装部署工具。OBD 同时也是包管理器，可以用来管理 OceanBase 所有的开源软件。本文介绍如何安装 OBD、使用 OBD 和 OBD 的命令。

obd部署oceanbase集群

在基础环境准备好之后，通过如下命令进行oceanbase集群发布，命令执行后会输出主要步骤提示，如下为obd执行deploy时，输出结果

[root@obcontrol obtools]# obd cluster deploy ob-cluster -c ./mini-distributed-with-obproxy-example.yaml
Package oceanbase-ce-3.1.0 is available.
Package obproxy-3.1.0 is available.
install oceanbase-ce-3.1.0 for local ok
install obproxy-3.1.0 for local ok
+-----------------------------------------------------------------------------+
|                                   Packages                                  |
+--------------+---------+---------+------------------------------------------+
| Repository   | Version | Release | Md5                                      |
+--------------+---------+---------+------------------------------------------+
| oceanbase-ce | 3.1.0   | 3.el7   | b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 |
| obproxy      | 3.1.0   | 1.el7   | 0b17cf0459a3b53c5a2febb6572894d183154c64 |
+--------------+---------+---------+------------------------------------------+
Repository integrity check ok
Parameter check ok
Open ssh connection ok
Remote oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository install ok
Remote oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository lib check !!
[WARN] z1(192.168.20.142) oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 require: libmariadb.so.3
[WARN] z2(192.168.20.143) oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 require: libmariadb.so.3
[WARN] z3(192.168.20.144) oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 require: libmariadb.so.3

Remote obproxy-3.1.0-0b17cf0459a3b53c5a2febb6572894d183154c64 repository install ok
Remote obproxy-3.1.0-0b17cf0459a3b53c5a2febb6572894d183154c64 repository lib check ok
Try to get lib-repository
Package oceanbase-ce-libs-3.1.0 is available.
install oceanbase-ce-libs-3.1.0 for local ok
Use oceanbase-ce-libs-3.1.0-528144ec7ff0194a8b326491a396b8f5c87b1eaa for oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9
Remote oceanbase-ce-libs-3.1.0-528144ec7ff0194a8b326491a396b8f5c87b1eaa repository install ok
Remote oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository lib check ok
Cluster status check ok
Initializes cluster work home ok
Initializes cluster work home ok
ob-cluster deployed

obd deploy日志分析

obd默认日志存放位置/root/.obd/log/obd，日志格式如下：

[2021-12-21 17:36:37] [7e950b54-6241-11ec-865e-00505691b9b1] [DEBUG] - mkdir /root/.obd/cluster/

[时间] [暂不清楚] [日志级别] [日志内容（步骤执行命令或信息输出）]   
其中日志级别DEBUG时，日志内容为执行命令
日志级别INFO时，日志内容为信息输出

从最后一列看，整个信息输出非常详细，只看INFO类型的信息可以发现，和obd deploy执行时的输出一样

[root@obcontrol log]#    grep INFO obd_deploy |awk -F']' '{print $4}'
 Package oceanbase-ce-3.1.0 is available.
 Package obproxy-3.1.0 is available.
 install oceanbase-ce-3.1.0 for local
 install obproxy-3.1.0 for local
 +-----------------------------------------------------------------------------+
 Repository integrity check
 Parameter check
 Open ssh connection
 Remote oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository install
 Remote oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository lib check
 [WARN
 Remote obproxy-3.1.0-0b17cf0459a3b53c5a2febb6572894d183154c64 repository install
 Remote obproxy-3.1.0-0b17cf0459a3b53c5a2febb6572894d183154c64 repository lib check
 Try to get lib-repository
 Package oceanbase-ce-libs-3.1.0 is available.
 install oceanbase-ce-libs-3.1.0 for local
 Use oceanbase-ce-libs-3.1.0-528144ec7ff0194a8b326491a396b8f5c87b1eaa for oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9
 Remote oceanbase-ce-libs-3.1.0-528144ec7ff0194a8b326491a396b8f5c87b1eaa repository install
 Remote oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository lib check
 Cluster status check
 Initializes cluster work home
 Initializes cluster work home
 ob-cluster deployed

下面针对每一步的信息输出做简要分析。

步骤 Package oceanbase-ce-3.1.0 is available

以下是在输出“Package oceanbase-ce-3.1.0 is available”时，obd执行信息，主要分为两部分

[2021-12-21 17:37:02]  [DEBUG] - Get Deploy by name <======step 1
...
[2021-12-21 17:37:02]  [DEBUG] - mkdir /root/.obd/cluster/ob-cluster  
[2021-12-21 17:37:02]  [DEBUG] - copy ./mini-distributed-with-obproxy-example.yaml /root/.obd/cluster/ob-cluster/config.yaml 
[2021-12-21 17:37:02]  [DEBUG] - Get deploy configuration
...
[2021-12-21 17:37:02]  [DEBUG] - Get oceanbase-ce repository <======step 2
[2021-12-21 17:37:02]  [DEBUG] - mkdir /root/.obd/repository
[2021-12-21 17:37:02]  [DEBUG] - mkdir /root/.obd/repository/oceanbase-ce 
[2021-12-21 17:37:02]  [DEBUG] - Search oceanbase-ce package from mirror
[2021-12-21 17:37:02]  [DEBUG] - mkdir /root/.obd/mirror
[2021-12-21 17:37:02]  [DEBUG] - mkdir /root/.obd/mirror/remote
[2021-12-21 17:37:02]  [DEBUG] - mkdir /root/.obd/mirror/local
[2021-12-21 17:37:02]  [DEBUG] - md5 is None
[2021-12-21 17:37:02]  [DEBUG] - name is oceanbase-ce
.....
[2021-12-21 17:37:02]  [DEBUG] - get RPM package by name: oceanbase-ce
version: 3.1.0
release:3.el7
arch: x86_64
md5: b73bcd531bdf3f087391991b290ff2cbcdaa0dc9
[2021-12-21 17:37:02]  [DEBUG] - Found Package oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9
[2021-12-21 17:37:02]  [INFO] Package oceanbase-ce-3.1.0 is available.
  • step 1 :集群信息存储，通过ob-cluster创建集群目录，然后把yaml文件复制到对应目录
[root@obcontrol ob-cluster]# cd /root/.obd/cluster/ob-cluster
[root@obcontrol ob-cluster]# ll
total 4
-rw-r--r-- 1 root root 2269 Dec 21 17:37 config.yaml
  • step 2 :分别在remote和local 2个目录，寻找oceanbase-ce包
[root@obcontrol local]# pwd
/root/.obd/mirror/local
[root@obcontrol local]# ll
total 53560
-rw-r--r-- 1 root root  8074612 Dec 21 17:29 obproxy-3.1.0-1.el7.x86_64.rpm
-rw-r--r-- 1 root root 46606940 Dec 21 17:29 oceanbase-ce-3.1.0-3.el7.x86_64.rpm
-rw-r--r-- 1 root root   158972 Dec 21 17:29 oceanbase-ce-libs-3.1.0-3.el7.x86_64.rpm

至此Package oceanbase-ce-3.1.0 is available步骤执行结束。

步骤Package obproxy-3.1.0 is available

以下是在输出“Package obproxy-3.1.0 is available”时，obd执行信息，与发现oceanbase-ce步骤类似

[2021-12-21 17:37:02]  [DEBUG] - Get obproxy repository <==== step 1
[2021-12-21 17:37:02]  [DEBUG] - mkdir /root/.obd/repository/obproxy
[2021-12-21 17:37:02]  [DEBUG] - Search obproxy package from mirror
[2021-12-21 17:37:02]  [DEBUG] - md5 is None
[2021-12-21 17:37:02]  [DEBUG] - name is obproxy
[2021-12-21 17:37:02]  [DEBUG] - arch is ['ia32e', 'x86_64', 'athlon', 'i686', 'i586', 'i486', 'i386', 'noarch']
[2021-12-21 17:37:02]  [DEBUG] - release is None
[2021-12-21 17:37:02]  [DEBUG] - version is None
[2021-12-21 17:37:02]  [DEBUG] - get RPM package by name: obproxy
version: 3.1.0
release:1.el7
arch: x86_64
md5: 0b17cf0459a3b53c5a2febb6572894d183154c64
[2021-12-21 17:37:02]  [DEBUG] - Found Package obproxy-3.1.0-0b17cf0459a3b53c5a2febb6572894d183154c64
[2021-12-21 17:37:02]  [INFO] Package obproxy-3.1.0 is available.
  • step 1 :分别在remote和local 2个目录，寻找oceanbase-ce包
[root@obcontrol local]# pwd
/root/.obd/mirror/local
[root@obcontrol local]# ll
total 53560
-rw-r--r-- 1 root root  8074612 Dec 21 17:29 obproxy-3.1.0-1.el7.x86_64.rpm
-rw-r--r-- 1 root root 46606940 Dec 21 17:29 oceanbase-ce-3.1.0-3.el7.x86_64.rpm
-rw-r--r-- 1 root root   158972 Dec 21 17:29 oceanbase-ce-libs-3.1.0-3.el7.x86_64.rpm

至此Package obproxy-3.1.0 is available步骤执行结束。

步骤install oceanbase-ce-3.1.0/obproxy-3.1.0 for local ok

以下是在输出“install oceanbase-ce-3.1.0/obproxy-3.1.0 for local ok”时，obd执行信息，目的主要是把rpm解压至本地的制定目录。

[2021-12-21 17:37:02]  [DEBUG] - Searching install plugin for components ...
...
[2021-12-21 17:37:02]  [DEBUG] - Search install plugin for oceanbase-ce
[2021-12-21 17:37:02]  [DEBUG] - mkdir /root/.obd/plugins
[2021-12-21 17:37:02]  [DEBUG] - Found for oceanbase-ce-install-3.1.0 for oceanbase-ce-3.1.0 
[2021-12-21 17:37:02]  [DEBUG] - Search install plugin for obproxy
[2021-12-21 17:37:02]  [DEBUG] - Found for obproxy-install-3.1.0 for obproxy-3.1.0
...
[2021-12-21 17:37:08]  [DEBUG] - extract ./home/admin/oceanbase/bin/observer to /root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9/bin/observer  <==== step 1
...
[2021-12-21 17:37:14]  [DEBUG] - extract ./home/admin/obproxy-3.1.0/bin/obproxy to /root/.obd/repository/obproxy/3.1.0/0b17cf0459a3b53c5a2febb6572894d183154c64/bin/obproxy  <==== step 2
...
  • step 1:把oceanbase-ce包，通过plugins文件目录下的py脚本进行解压至对应目录
[root@obcontrol 3.1.0]# pwd
/root/.obd/plugins/oceanbase-ce/3.1.0
[root@obcontrol 3.1.0]# ll
total 188
-rw-r--r-- 1 root root  3432 Dec 21 17:29 bootstrap.py
-rw-r--r-- 1 root root  2564 Dec 21 17:29 connect.py
....
[root@obcontrol repository]# ll
total 0
drwxr-xr-x 3 root root 19 Dec 21 17:37 obproxy
drwxr-xr-x 3 root root 19 Dec 21 17:37 oceanbase-ce
drwxr-xr-x 3 root root 19 Dec 21 17:37 oceanbase-ce-libs
[root@obcontrol repository]# tree -L 2 oceanbase-ce
oceanbase-ce
└── 3.1.0
    ├── b73bcd531bdf3f087391991b290ff2cbcdaa0dc9
    └── oceanbase-ce -> /root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9

3 directories, 0 files
# 以上目录信息与/root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9输出信息一致
  • step 2:把oceanbase-ce包，通过plugins文件目录下的py脚本进行解压至对应目录
[root@obcontrol repository]# ll
total 0
drwxr-xr-x 3 root root 19 Dec 21 17:37 obproxy
drwxr-xr-x 3 root root 19 Dec 21 17:37 oceanbase-ce
drwxr-xr-x 3 root root 19 Dec 21 17:37 oceanbase-ce-libs
[root@obcontrol repository]# tree -L 2 obproxy
obproxy
└── 3.1.0
    ├── 0b17cf0459a3b53c5a2febb6572894d183154c64
    └── obproxy -> /root/.obd/repository/obproxy/3.1.0/0b17cf0459a3b53c5a2febb6572894d183154c64

3 directories, 0 files
# 以上目录信息与/root/.obd/repository/obproxy/3.1.0/0b17cf0459a3b53c5a2febb6572894d183154c64输出信息一致

至此install oceanbase-ce-3.1.0/obproxy-3.1.0 for local ok步骤执行结束。

步骤Repository integrity check ok

此步骤日志输出信息较少，猜测应该是通过plugin文件目录的内容来检查，解压出来oceanbase-ce和obproxy的完整性。

步骤Parameter check ok

以下是"Parameter check ok"信息输出

[2021-12-21 17:37:16]  [INFO] Parameter check <=== step 1
[2021-12-21 17:37:16]  [DEBUG] - Searching param plugin for components ...
[2021-12-21 17:37:16]  [DEBUG] - Search param plugin for oceanbase-ce
[2021-12-21 17:37:16]  [DEBUG] - Found for oceanbase-ce-param-3.1.0 for oceanbase-ce-3.1.0
[2021-12-21 17:37:16]  [DEBUG] - Applying oceanbase-ce-param-3.1.0 for oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9
[2021-12-21 17:37:17]  [DEBUG] - Search param plugin for obproxy
[2021-12-21 17:37:17]  [DEBUG] - Found for obproxy-param-3.1.0 for obproxy-3.1.0
[2021-12-21 17:37:17]  [DEBUG] - Applying obproxy-param-3.1.0 for obproxy-3.1.0-0b17cf0459a3b53c5a2febb6572894d183154c64
...
  • step 1：利用plugin中的param来检查解压文件的正确性，但不清楚其具体原理

步骤Open ssh connection ok

此步骤比较重要，应该是整个集群部署的关键，

[2021-12-21 17:37:17]  [INFO] Open ssh connection
[2021-12-21 17:37:18]  [INFO] Remote oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository install
[2021-12-21 17:37:18]  [DEBUG] - Remote oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository integrity check
[2021-12-21 17:37:18]  [DEBUG] - z1(192.168.20.142) oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository integrity check  <====step 1
[2021-12-21 17:37:18]  [DEBUG] - root@192.168.20.142 execute: echo $HOME/.obd 
[2021-12-21 17:37:18]  [DEBUG] - root@192.168.20.142 execute: cat /root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9/.data 
[2021-12-21 17:37:18]  [DEBUG] - exited code 1, error output:
cat: /root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9/.data: No such file or directory
[2021-12-21 17:37:18]  [DEBUG] - z1(192.168.20.142) oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 install check
[2021-12-21 17:37:18]  [DEBUG] - z1(192.168.20.142) oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 need to be installed 
[2021-12-21 17:37:18]  [DEBUG] - z1(192.168.20.142) oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 installing
[2021-12-21 17:37:18]  [DEBUG] - root@192.168.20.142 execute: mkdir -p /root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9/bin 
[2021-12-21 17:37:18]  [DEBUG] - exited code 0
[2021-12-21 17:37:27]  [DEBUG] - root@192.168.20.142 execute: chmod 755 /root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9/bin/observer 
[2021-12-21 17:37:27]  [DEBUG] - exited code 0
[2021-12-21 17:37:27]  [DEBUG] - root@192.168.20.142 execute: mkdir -p /root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 
[2021-12-21 17:37:27]  [DEBUG] - exited code 0
[2021-12-21 17:37:27]  [DEBUG] - z1(192.168.20.142) oceanbase-ce installed
...
[2021-12-21 17:37:46]  [INFO] Remote oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository lib check
[2021-12-21 17:37:46]  [DEBUG] - z1(192.168.20.142) oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 repository lib check  <===step 2
[2021-12-21 17:37:46]  [DEBUG] - root@192.168.20.142 execute: echo $HOME/.obd 
[2021-12-21 17:37:46]  [DEBUG] - exited code 0
[2021-12-21 17:37:46]  [DEBUG] - root@192.168.20.142 set env LD_LIBRARY_PATH to '/root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9/lib:'
[2021-12-21 17:37:46]  [DEBUG] - root@192.168.20.142 execute: ldd /root/.obd/repository/oceanbase-ce/3.1.0/b73bcd531bdf3f087391991b290ff2cbcdaa0dc9/bin/observer 
[2021-12-21 17:37:46]  [DEBUG] - exited code 0
[2021-12-21 17:37:46]  [WARNING] [WARN] z1(192.168.20.142) oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 require: libmariadb.so.3
...
  • step1 ：通过之前配置的免密，将oceanbase-ce包传输至集群节点的/root/.obd目录
[root@ob01 repository]# ll
total 0
drwxr-xr-x 3 root root 19 Dec 21 17:37 oceanbase-ce
drwxr-xr-x 3 root root 19 Dec 21 17:37 oceanbase-ce-libs
  • step2：校验oceanbase-ce所需的lib包，发现不存在，需要从中控机寻找

步骤oceanbase-ce-libs

此步骤执行和步骤3.1、3.3、3.6类似，不做过多的介绍。

步骤Cluster status check

此步骤内容较为简单，就是检查集群状态，通过验证此/root/observer目录

[2021-12-21 17:37:54]  [INFO] Cluster status check
[2021-12-21 17:37:54]  [DEBUG] - Searching status plugin for components ...
[2021-12-21 17:37:54]  [DEBUG] - Searching status plugin for oceanbase-ce-3.1.0-b73bcd531bdf3f087391991b290ff2cbcdaa0dc9
[2021-12-21 17:37:54]  [DEBUG] - Found for oceanbase-ce-py_script_status-3.1.0 for oceanbase-ce-3.1.0
...
[2021-12-21 17:37:54]  [DEBUG] - import status
[2021-12-21 17:37:54]  [DEBUG] - add status ref count to 1
[2021-12-21 17:37:54]  [DEBUG] -- root@192.168.20.142 execute: cat /root/observer/run/observer.pid 
...

步骤Initializes cluster work home ok

此步骤在/root/observer 目录创建oceanserver所需的目录，所有节点均执行，这样做标准化。

[2021-12-21 17:37:55]  [DEBUG] - sub status ref count to 0
...
[2021-12-21 17:37:55]  [INFO] Initializes cluster work home
[2021-12-21 17:37:55]  [DEBUG] -- root@192.168.20.142 execute: echo $HOME/.obd 
[2021-12-21 17:37:55]  [DEBUG] -- exited code 0
[2021-12-21 17:37:55]  [DEBUG] -- z1(192.168.20.142) initializes cluster work home
[2021-12-21 17:37:55]  [DEBUG] -- root@192.168.20.142 execute: mkdir -p /root/observer 
[2021-12-21 17:37:55]  [DEBUG] -- exited code 0
[2021-12-21 17:37:55]  [DEBUG] -- root@192.168.20.142 execute: ls /root/observer 
[2021-12-21 17:37:55]  [DEBUG] -- exited code 0
[2021-12-21 17:37:55]  [DEBUG] -- root@192.168.20.142 execute: bash -c "mkdir -p /root/observer/{etc,admin,.conf,log,bin,lib}" 
...

[2021-12-21 17:38:03]  [INFO] Initializes cluster work home

步骤ob-cluster deployed

此步骤主要是将发布信息记录到相关文件中。

[2021-12-21 17:38:03]  [DEBUG] - dump deploy info to /root/.obd/cluster/ob-cluster/.data
[2021-12-21 17:38:03]  [INFO] ob-cluster deployed

如下：

[root@obcontrol log]# cat /root/.obd/cluster/ob-cluster/.data
name: ob-cluster
components:
  oceanbase-ce:
    hash: b73bcd531bdf3f087391991b290ff2cbcdaa0dc9
    version: 3.1.0
  obproxy:
    hash: 0b17cf0459a3b53c5a2febb6572894d183154c64
    version: 3.1.0
status: STATUS_DEPLOYED
config_status: UNCHNAGE

obd deploy总结

通过以上分析，deploy主要步骤如下：

1、oceanbase-ce/obproxy 本地节点安装

2、文件、参数一致性检查

3、oceanbase-ce/obproxy 远程节点安装

4、oceanbase-ce-lib检查及安装

5、集群节点目录初始化

6、deploy步骤归档

当然有关obd cluster start/stop/destroy的日志信息也很详细，可以通过如上方法进行解析，以便更好的了解obd运行过程。

oceanbase征文
「喜欢这篇文章，您的关注和赞赏是给作者最好的鼓励」
关注作者
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文章的来源（墨天轮），文章链接，文章作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

文章被以下合辑收录

课程笔记 OceanBase 社区版入门到实战（共6篇）
《OceanBase 社区版入门到实战》的课程笔记

评论