oceanbase日常使用中,默认情况下日志产生较快,日志目录较小时,如果不做任何处理,将会很快导致日志目录撑满,从而导致服务异常。


0.ENV

oceanbase企业版、oceanbase社区版
当前采用单机3个ob server部署,生成环境不建议该部署方式。

1. 现象

oceanbase数据库主机,/home目录使用率100%。
[admin@ob3 ~]$ df -Th homeFilesystem Type Size Used Avail Use% Mounted on/dev/mapper/centos-home ext4 30G 30G 0G 100% home

2. 问题分析

当前测试环境,因为我们实验环境留给log的空间比较小,/home共30G,通过目录大小查看,observer3 log目录使用13G,占用空间较大。
[admin@ob3 ~]$ du -sh *4.0K 1host-3ob-cluster.yaml28G oceanbase[admin@ob3 ~]$ cd oceanbase/[admin@ob3 oceanbase]$ ls1 2 3[admin@ob3 oceanbase]$ du -sh *7.9G 17.5G 213G 3[admin@ob3 oceanbase]$ du -sh 3/*4.0K 3/admin12K 3/etc12K 3/etc212K 3/etc313G 3/log8.0K 3/run0 3/store

3. 问题处理

以sys租户登录,启动日志文件循环,并且只保留10个日志文件。
打开日志循环,设置为true。该值默认为false,即关闭日志循环。
obclient> alter system set enable_syslog_recycle=true;Query OK, 0 rows affected (0.09 sec)
查看日志循环已经打开,value一栏3个zone均为True。
MySQL [(none)]> show parameters like '%enable_syslog_recycle%';+-------+----------+-----------+----------+-----------------------+-----------+-------+---------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+| zone | svr_type | svr_ip | svr_port | name | data_type | value | info | section | scope | source | edit_level |+-------+----------+-----------+----------+-----------------------+-----------+-------+---------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+| zone3 | observer | 127.0.0.1 | 4882 | enable_syslog_recycle | NULL | True | specifies whether log file recycling is turned on. Value: True:turned on; False: turned off | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE || zone2 | observer | 127.0.0.1 | 3882 | enable_syslog_recycle | NULL | True | specifies whether log file recycling is turned on. Value: True:turned on; False: turned off | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE || zone1 | observer | 127.0.0.1 | 2882 | enable_syslog_recycle | NULL | True | specifies whether log file recycling is turned on. Value: True:turned on; False: turned off | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |+-------+----------+-----------+----------+-----------------------+-----------+-------+---------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+3 rows in set (0.016 sec)
设置日志数目为10,即只保留10个日志文件,1个日志文件最大为256M。
obclient> alter system set max_syslog_file_count=10;Query OK, 0 rows affected (0.08 sec)
查看日志数目为10,value一栏3个zone均为10。
MySQL [(none)]> show parameters like '%max_syslog_file_count%';+-------+----------+-----------+----------+-----------------------+-----------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+| zone | svr_type | svr_ip | svr_port | name | data_type | value | info | section | scope | source | edit_level |+-------+----------+-----------+----------+-----------------------+-----------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+| zone2 | observer | 127.0.0.1 | 3882 | max_syslog_file_count | NULL | 10 | specifies the maximum number of the log files that can co-exist before the log file recycling kicks in. Each log file can occupy at most 256MB disk space. When this value is set to 0, no log file will be removed. Range: [0, +∞) in integer | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE || zone3 | observer | 127.0.0.1 | 4882 | max_syslog_file_count | NULL | 10 | specifies the maximum number of the log files that can co-exist before the log file recycling kicks in. Each log file can occupy at most 256MB disk space. When this value is set to 0, no log file will be removed. Range: [0, +∞) in integer | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE || zone1 | observer | 127.0.0.1 | 2882 | max_syslog_file_count | NULL | 10 | specifies the maximum number of the log files that can co-exist before the log file recycling kicks in. Each log file can occupy at most 256MB disk space. When this value is set to 0, no log file will be removed. Range: [0, +∞) in integer | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |+-------+----------+-----------+----------+-----------------------+-----------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+3 rows in set (0.016 sec)
设置完成后,日志会自动被清理,此时空间会被释放。
[admin@ob3 ~]$ df -Th homeFilesystem Type Size Used Avail Use% Mounted on/dev/mapper/centos-home ext4 30G 16G 13G 56% home
查看observer日志:
[admin@ob3 ~]$ cd oceanbase/3/log[admin@ob3 log]$ du -sh *154M election.log257M election.log.20210717135834257M election.log.20210717142732129M election.log.wf196M election.log.wf.20210717135834225M election.log.wf.20210717142732210M observer.log257M observer.log.20210717094420257M observer.log.20210717103300257M observer.log.20210717111741257M observer.log.20210717121149257M observer.log.20210717125650257M observer.log.20210717132204257M observer.log.20210717135635257M observer.log.20210717140656257M observer.log.20210717142621257M observer.log.2021071714395850M observer.log.wf11M observer.log.wf.2021071709442011M observer.log.wf.2021071710330021M observer.log.wf.2021071711174113M observer.log.wf.2021071712114938M observer.log.wf.2021071712565055M observer.log.wf.2021071713220445M observer.log.wf.2021071713563592M observer.log.wf.2021071714065668M observer.log.wf.2021071714262178M observer.log.wf.20210717143958131M rootservice.log257M rootservice.log.20210716232113257M rootservice.log.20210717034416257M rootservice.log.20210717075924257M rootservice.log.20210717120926257M rootservice.log.2021071714203494M rootservice.log.wf7.7M rootservice.log.wf.202107162321131.9M rootservice.log.wf.202107170344162.3M rootservice.log.wf.202107170759249.4M rootservice.log.wf.20210717120926135M rootservice.log.wf.20210717142034

4. 小结

log增长过快而导致日志目录/home撑满,通过启动日志文件循环,并且只保留10个日志文件,自动清理日志,建议生产环境安装完成后,设置参数,消除隐患。也可设置日志级别为ERROR和WARN,减少日志输出,本文未做描述。
更多oceanbase日志参考:
https://www.oceanbase.com/docs/oceanbase-database/oceanbase-database-1-4/V1.4/oblog
不足之处,还望抛砖。
作者:王坤,微信公众号:rundba,欢迎转载,转载请注明出处。
如需公众号转发,请联系wx: landnow。
文章转载自rundba,如果涉嫌侵权,请发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。




