oceanbase初学习之手动部署 OceanBase 单副本集群

原创 null 2023-09-01

1845

1.系统环境配置

官方建议内存至少12G，我这电脑目前还有其他吃内存的再跑，就先设置11G了，从虚拟机上去看，目前是有10G内存；

参考地址：

https://open.oceanbase.com/blog/8600151

2.准备及搭建

1）安装RPM包

rpm包下载地址：https://www.oceanbase.com/softwarecenter

这里我选择的是4.0的版本，总计87M，包如下：

所有rpm包无需其他依赖，直接安装即可

rpm -ivh *
正在升级/安装...
   1:oceanbase-ce-libs-4.0.0.0-1000002################################# [ 50%]
   2:oceanbase-ce-4.0.0.0-100000272022################################# [100%]   3:libobclient-2.2.2-3.el7          ################################# [100%]
   4:obclient-2.2.2-1.el7             ################################# [100%]
   5:obproxy-ce-4.0.0-5.el7           ################################# [100%]

2）创建所需目录

mkdir /data /redo   ##生产环境要求这两个文件系统尽可能是两块独立的物理盘，或者最低要求是两个独立的逻辑盘chown -R admin:admin /data /redo
##问题############################################################
上述rpm包用root装完以后，存在/home/admin/oceanabse的目录，且属主为root，需要修改
sudo chown -R admin:admin /home/admin/oceanbase/
##################################################################
su - adminmkdir -p ~/oceanbase/store/obdemo       #集群目录mkdir -p /data/obdemo/{sstable,etc3} mkdir -p /redo/obdemo/{clog,etc2,ilog,slog}
之后做软链接：
ln -s /redo/obdemo/* /home/admin/oceanbase/store/obdemo/
ln -s /data/obdemo/* /home/admin/oceanbase/store/obdemo/
完成之后展示如下：[admin@obce02 ~]$ tree ~/oceanbase/store/ /data/ /redo/
/home/admin/oceanbase/store/
└── obdemo
    ├── clog -> /redo/obdemo/clog
    ├── etc2 -> /redo/obdemo/etc2
    ├── etc3 -> /data/obdemo/etc3
    ├── ilog -> /redo/obdemo/ilog
    ├── slog -> /redo/obdemo/slog
    └── sstable -> /data/obdemo/sstable
/data/
└── obdemo
    ├── etc3
    └── sstable
/redo/
└── obdemo
    ├── clog
    ├── etc2
    ├── ilog
    └── slog

3）配置环境变量

vim /etc/security/limits.confroot soft nofile 655350
root hard nofile 655350
* soft nofile 655350
* hard nofile 655350
* soft stack 20480
* hard stack 20480
* soft nproc 655360
* hard nproc 655360
* soft core unlimited
* hard core unlimited

vi /etc/sysctl.conf(完成后执行sysctl -p)
# for oceanbase
## 修改内核异步 I/O 限制
fs.aio-max-nr=1048576

## 网络优化
net.core.somaxconn = 2048
net.core.netdev_max_backlog = 10000 
net.core.rmem_default = 16777216 
net.core.wmem_default = 16777216 
net.core.rmem_max = 16777216 
net.core.wmem_max = 16777216

net.ipv4.ip_local_port_range = 3500 65535 
net.ipv4.ip_forward = 0 
net.ipv4.conf.default.rp_filter = 1 
net.ipv4.conf.default.accept_source_route = 0 
net.ipv4.tcp_syncookies = 0 
net.ipv4.tcp_rmem = 4096 87380 16777216 
net.ipv4.tcp_wmem = 4096 65536 16777216 
net.ipv4.tcp_max_syn_backlog = 16384 
net.ipv4.tcp_fin_timeout = 15 
net.ipv4.tcp_max_syn_backlog = 16384 
net.ipv4.tcp_tw_reuse = 1 
net.ipv4.tcp_tw_recycle = 1 
net.ipv4.tcp_slow_start_after_idle=0

vm.swappiness = 0
vm.min_free_kbytes = 2097152

# 此处为 OceanBase 数据库的 data 目录
kernel.core_pattern = /data/core-%e-%p-%t

4）启动observer进程

echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/oceanbase/lib' >> ~/.bash_profile
source ~/.bash_profile      #配置环境变量###上述一定要进行配置，不然会出现./observer: error while loading shared libraries: libmariadb.so.3: cannot open shared object file: No such file or directory，找不到指定lib

#注意下面的datafile_size设置的是50G，我没有这么多空间，修改了几次一直报错，目前设置的30Gcd ~/oceanbase  ##使用admin用户进入到启动目录
bin/observer -i ens38 -p 2881 -P 2882 -z zone1 -d ~/oceanbase/store/obdemo -r '192.168.61.141:2882:2881' -c 20230901 -n obdemo -o "memory_limit=8G,cache_wash_threshold=1G,__min_full_resource_pool_memory=268435456,system_memory=3G,memory_chunk_cache_size=128M,cpu_count=4,net_thread_count=2,datafile_size=50G,stack_size=1536K,config_additional_dir=/data/obdemo/etc3;/redo/obdemo/etc2"-i：指定网卡名-d：数据目录-p：数据库端口
-P:远程访问协议端口号-c:集群id-n：集群名称-o：相关配置参数cpu_count=4,net_thread_count=2  #根据自己情况设置

！！检查进程没启动成功
查看日志发现有个配置报错，对应调整到2G（__min_full_resource_pool_memory=2147483648）[root@obcp ~]less /home/admin/oceanbase/log/observer.log
[2023-09-01 13:18:15.152536] ERROR [SHARE] operator() (ob_common_config.cpp:124) [22206][][T0][Y0-0000000000000000-0-0] [lt=9] Invalid config, value out of range(name="__min_full_resource_pool_memory", value="268435456", ret=-4147) BACKTRACE:0xb553efb 0xb5459d6 0x3c3bfda 0x3c3bcf9 0x3c3bb00 0x3c3b952 0x9a21ee4 0x99fa59f 0x99f9d88 0x5bd2d95 0x5bd1386 0x3c173fc 0x7fe733925495 0x3c16184q启动还是报错！
再次查看日志发现，磁盘空间不够了ERROR [SERVER] init (ob_server.cpp:374) [22647][][T0][Y0-0000000000000000-0-0] [lt=4] [OBSERVER_NOTICE] fail to init observer(ret=-4184, ret="OB_SERVER_OUTOF_DISK_SPACE") BACKTRACE:0xb553efb 0xb5459d6 0x3c42f9b 0x3c42cb4 0x3c42ac9 0x3c2137b 0x5bdd615 0x5bd1f27 0x3c173fc 0x7fd04d4e6495 0x3c16184ook手动lvm扩容下

再次执行就成功了。（将datafile_size缩小到30G了，最开始写的50G，结果redo+datafile的值超了，一直报错，改了几次才可以，目前剩余12G）

5）bootstrap集群自举（初始化）

obclient -h127.0.0.1 -uroot@sys -P2881 -p -A -c   #登陆（MySQL客户端也可以登陆）###此时show databases;  没有数据库，会报错，需要执行下面的初始化语句root@db 15:07:  [(none)]>set session ob_query_timeout=1000000000;alter system bootstrap ZONE 'zone1' SERVER '192.168.61.141:2882';Query OK, 0 rows affected (0.014 sec)
Query OK, 0 rows affected (51.968 sec)

##再次查看数据库就可以查到了root@db 15:12:  [(none)]>show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| LBACSYS            |
| mysql              |
| oceanbase          |
| ORAAUDITOR         |
| SYS                |
| test               |
+--------------------+
7 rows in set (0.018 sec)

##如果是集群，自检不成功需要确认如下内容###############################集群节点之间时间同步延时超过 50ms 
集群节点之间网络延时超过 100ms 。
集群节点上 OBSERVER 相关目录结构不对或者目录权限不对。
集群节点上进程 observer 启动参数写的不对。注意隐含参数的名字（如__min_full_resource_pool_memory )、参数 -d 的目录是否正确、参数 -z跟 IP 的对应关系、 参数中多了空格或分隔符错误（有的是 , ，有的是;）。
集群节点可用内存低于进程 observer 启动参数 memory_limit 值。

设置root用户密码：alter user root identified by 'rootPWD123' ;

OBPROXY 用户（proxyro）密码默认 OBPROXY 连接 OceanBase 集群使用用户 proxyro 。这个用户不存在，需要创建。
grant select on oceanbase.* to proxyro identified by 'rootPWD123' ;

6）安装/运行obproxy

[admin@obcp ~]$ ll
总用量 0
drwxr-xr-x  3 admin admin  17 9月   1 10:51 obproxy-4.0.0   # 之前装的rpm包，会在这里创建文件夹

##启动obproxy[admin@obcp bin]$ cd ~/obproxy-4.0.0/ && bin/obproxy -r "192.168.61.141:2881" -p 2883 -o "enable_strict_kernel_release=false,enable_cluster_checkout=false,enable_metadb_used=false" -c obdemo

[admin@obcp obproxy-4.0.0]$ ll   #安装完成会生成这么多文件
总用量 0
drwxr-xr-x 2 admin admin  40 9月   1 10:51 bin
drwxrwxr-x 2 admin admin   6 9月   1 15:45 control-config
drwxrwxr-x 2 admin admin  84 9月   1 15:46 etc
drwxrwxr-x 2 admin admin 301 9月   1 15:45 log
drwxrwxr-x 2 admin admin   6 9月   1 15:45 sharding-config

检查进程，存在就成功了
再确认下端口：[admin@obcp obproxy-4.0.0]$ netstat -ntlp |grep obproxy
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:2883            0.0.0.0:*               LISTEN      16554/bin/obproxy   
tcp        0      0 0.0.0.0:2884            0.0.0.0:*               LISTEN      16554/bin/obproxy   很好：2883/2884都存在

登陆obproxy：
[admin@obcp obproxy-4.0.0]$ obclient -h 127.0.0.1 -u root@proxysys -P 2883      #不用输入密码
ERROR 2027 (HY000): received malformed packet
##报错了，在论坛查了查，这是之前的一个bug，可以用mysql客户端登陆，使用obclient登陆会报错##目前官方已经解决了，在后边追加--proxy-mode即可登陆
[admin@obcp obproxy-4.0.0]$ obclient -h 127.0.0.1 -u root@proxysys -P 2883 --proxy-mode    #成功登陆Welcome to the OceanBase.  Commands end with ; or \g.
Your OceanBase connection id is 3
Server version: 

Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.


root@proxysys@db 16:00:  [(none)]>show proxyconfig like '%sys_password%';
+------------------------+-------+--------------------------------+-------------+---------------+
| name                   | value | info                           | need_reboot | visible_level |
+------------------------+-------+--------------------------------+-------------+---------------+
| observer_sys_password1 |       | password for observer sys user | false       | SYS           |
| observer_sys_password  |       | password for observer sys user | false       | SYS           |
| obproxy_sys_password   |       | password for obproxy sys user  | false       | SYS           |
+------------------------+-------+--------------------------------+-------------+---------------+

修改 OBPROXY 用户密码是通过修改参数的方式，命令是：alter proxyconfig set 
root@proxysys@db 15:50:  [(none)]>  alter proxyconfig set observer_sys_password = 'rootPWD123';

同时还需要修改 OBPROXY 连接OceanBase集群用户proxyro的密码。这样 OBPROXY才能跟OceanBase集群正常连接。
alter proxyconfig set observer_sys_password = 'rootPWD123' ; 
##这个密码就是前面 OceanBase 集群初始化后创建的用户 proxyro 的密码。

之后使用密码再次进行登陆，可以正常登陆，成功！
[admin@obcp obproxy-4.0.0]$ obclient -h 127.0.0.1 -u root@sys -P 2883  -p -A -c
Enter password: 
Welcome to the OceanBase.  Commands end with ; or \g.
Your OceanBase connection id is 12
Server version: OceanBase_CE 4.0.0.0 (r100000272022110114-6af7f9ae79cd0ecbafd4b1b88e2886ccdba0c3be) (Built Nov  1 2022 14:57:18)

Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

root@db 16:09:  [(none)]>show processlist;
+------+--------+------+-----------------+------+-------------+-------------------+-------------------+-------+-------+
| Id   | Tenant | User | Host            | db   | trans_count | svr_session_count | state             | tid   | pid   |
+------+--------+------+-----------------+------+-------------+-------------------+-------------------+-------+-------+
|   12 | sys    | root | 127.0.0.1:41800 | NULL |           0 |                 1 | MCS_ACTIVE_READER | 16554 | 16554 |
+------+--------+------+-----------------+------+-------------+-------------------+-------------------+-------+-------+
1 row in set (0.008 sec)

root@db 16:09:  [(none)]>

3.创建租户/数据库并测试

1.设置参数

alter resource unit sys_unit_config min_cpu=5;  

报错：ERROR 1210 (HY000): Incorrect arguments to MIN_CPU, MIN_CPU is greater than MAX_CPU

root@db 16:48: [oceanbase]>alter resource unit sys_unit_config max_cpu=2;
Query OK, 0 rows affected (0.029 sec)

root@db 16:49: [oceanbase]>alter resource unit sys_unit_config min_cpu=2;
Query OK, 0 rows affected (0.009 sec)

root@db 16:49:  [oceanbase]> select * from __all_unit_config;
+-------------------------------+-------------------------------+----------------+-----------------+-----------+---------+----------------+--------------- +------------+----------+-------------+
| gmt_create                    | gmt_modified                  | unit_config_id | name            | max_cpu   | min_cpu | memory_size    | log_disk_size  |  max_iops  | min_iops | iops_weight |
+-------------------------------+-------------------------------+----------------+-----------------+-----------+---------+----------------+----------------+------------+----------+-------------+

| 2023-09-01 15:11:24.712504    | 2023-09-01 16:49:05.331749    | 1              | sys_unit_config |  2        |    2    | 2147483648     | 2147483648     |  10000     | 10000     |          1 |
+-------------------------------+-------------------------------+----------------+-----------------+-----------+---------+----------------+----------------+------------+-----------+-------------+

2.创建资源单元
root@db 16:49:  [oceanbase]> CREATE resource unit S4C1G max_cpu=1, min_cpu=1, memory_size='2G', max_iops=1024, min_iops=1024, log_disk_size='10G';##根据实际情况填写，我配置的cpu少Query OK, 0 rows affected (0.019 sec)
root@db 16:53:  [oceanbase]> select * from __all_unit_config; 
+----------------------------+----------------------------+----------------+-----------------+---------+---------+-------------+---------------+----------+----------+-------------+
| gmt_create                 | gmt_modified               | unit_config_id | name            | max_cpu | min_cpu | memory_size | log_disk_size | max_iops | min_iops | iops_weight |
+----------------------------+----------------------------+----------------+-----------------+---------+---------+-------------+---------------+----------+----------+-------------+
| 2023-09-01 15:11:24.712504 | 2023-09-01 16:49:05.331749 |              1 | sys_unit_config |       2 |       2 |  2147483648 |    2147483648 |    10000 |    10000 |           1 |
| 2023-09-01 16:53:20.777207 | 2023-09-01 16:53:20.777207 |           1001 | S4C1G           |       1 |       1 |  2147483648 |   10737418240 |     1024 |     1024 |           0 |
+----------------------------+----------------------------+----------------+-----------------+---------+---------+-------------+---------------+----------+----------+-------------+

3.创建资源池
root@db 16:54:  [oceanbase]> CREATE resource pool my_pool unit = 'S4C1G', unit_num = 1;
4.创建租户root@db 16:54:  [oceanbase> create tenant obmysql resource_pool_list=('my_pool'), primary_zone='RANDOM',comment 'mysql tenant/instance', charset='utf8' set ob_tcp_invited_nodes='%', ob_compatibility_mode='mysql'; exit;
Query 0K，0 rows affected (15.547 sec)
5.登录租户obmysql
[admin@obcp ~]$  obclient -h127.0.0.1 -P2881 -uroot@obmysql -p -Doceanbase -A -c
Welcome to the OceanBase.  Commands end with ; or \g.
Your OceanBase connection id is 3221487636
Server version: OceanBase_CE 4.0.0.0 (r100000272022110114-6af7f9ae79cd0ecbafd4b1b88e2886ccdba0c3be) (Built Nov  1 2022 14:57:18)
Copyright (c) 2000, 2018, OceanBase and/or its affiliates. All rights reserved.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
root@db 17:14:  [oceanbase]>

6.创建数据库root@db 17:17:  [oceanbase]> create database my_ob;
Query OK, 1 row affected (0.042 sec)

7.创建数据库表root@db 17:18:  [oceanbase]>use my_ob;
Database changed
root@db 17:18:  [my_ob]>create table test(id int primary key ,name varchar(32) );
Query OK, 0 rows affected (0.196 sec)

root@db 17:18:  [my_ob]>insert into test values(1,'张三'),(2,'李四 ');
Query OK, 2 rows affected (0.052 sec)
Records: 2  Duplicates: 0  Warnings: 0

root@db 17:19:  [my_ob]>select * from test;
+----+---------+
| id | name    |
+----+---------+
|  1 | 张三    |
|  2 | 李四    |
+----+---------+
2 rows in set (0.018 sec)

ok，第二个小实验完成，整体部署流程其实没有多大问题，就是在一些配置项上要注意，首先是存储一定要大（部署过程中涉及相关存储容量参数已标注），其次是内存设置要安装官方的来，我自己的内存还是小了，中间崩了一次；

然后就是obclient登陆obproxy 追加--proxy-mode参数，这个是在官方论坛看到的，中间其实遇到了好多错误，第一次手动搭建，还是有很多不足的；遇到错误多多查看官方论坛，收获不小！

oceanbase oceanbase 社区版

「喜欢这篇文章，您的关注和赞赏是给作者最好的鼓励」

关注作者