上期数据文件(sstable)刚进行过扩容(增大),但操作系统/home目录使用率达到80%,目前clog占用空间较大,仍存在在空间风险,通过调整clog参数,降低磁盘空间使用,保障observer运行。

0.ENV
CentOS 7.6;
OceanBase-ce 3.1.2。

1. 调整前信息查询
1.1 系统磁盘空间使用
当前/home使用率为80%
[admin@ob03 store]$ df -h | grep home/dev/mapper/centos-home 1.8T 1.4T 357G 80% /home
clog占用空间较大
[admin@ob03 store]$ du -sh *789G clog #clog存放目录,占用空间较大4.1G ilog145M slog623G sstable #数据存放目录

2. 调整clog磁盘空间使用
2.1 clog_disk_usage_limit_percentage参数说明
- name: clog_disk_usage_limit_percentagerequire: falsetype: INTdefault: 95min_value: 80max_value: 100section: TRANSneed_restart: falsedescription_en: maximum of clog disk usage percentage before stop submitting or receiving logs.description_local: 事务日志的磁盘IO最大可用的磁盘利用率
2.2 调小clog磁盘使用率
[admin@obp1 ~]$ obd cluster edit-config obcluster #添加参数,将clog磁盘空间使用率默认值80%降低为30%global:...clog_disk_utilization_threshold: 30...
2.3 使配置生效,不需要要重启,不影响业务运行
[admin@obp1 ~]$ obd cluster reload obclusterGet local repositories and plugins okLoad cluster param plugin okOpen ssh connection okCluster status check okConnect to obproxy okConnect to observer okobcluster reload
2.4 可进一步在数据库中查询,参数已生效
MySQL [oceanbase]> show parameters like 'clog_disk_utilization_threshold';+-------+----------+---------------+----------+---------------------------------+-----------+-------+---------------------------------------------------------------------------------------------------------------------------------+---------+---------+---------+-------------------+| zone | svr_type | svr_ip | svr_port | name | data_type | value | info | section | scope | source | edit_level |+-------+----------+---------------+----------+---------------------------------+-----------+-------+---------------------------------------------------------------------------------------------------------------------------------+---------+---------+---------+-------------------+| zone1 | observer | 192.168.8.242 | 2882 | clog_disk_utilization_threshold | NULL | 30 | clog disk utilization threshold before reuse clog files, should be less than clog_disk_usage_limit_percentage. Range: [10, 100) | TRANS | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE || zone3 | observer | 192.168.8.244 | 2882 | clog_disk_utilization_threshold | NULL | 30 | clog disk utilization threshold before reuse clog files, should be less than clog_disk_usage_limit_percentage. Range: [10, 100) | TRANS | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE || zone2 | observer | 192.168.8.243 | 2882 | clog_disk_utilization_threshold | NULL | 30 | clog disk utilization threshold before reuse clog files, should be less than clog_disk_usage_limit_percentage. Range: [10, 100) | TRANS | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |+-------+----------+---------------+----------+---------------------------------+-----------+-------+---------------------------------------------------------------------------------------------------------------------------------+---------+---------+---------+-------------------+3 rows in set (0.116 sec)

3. 调整后信息查询
3.1 系统磁盘空间使用
会发现clog目录会一直在减小,最终大小为64M。
[admin@ob01 store]$ du -sh *768G clog4.0G ilog96M slog623G sstable[admin@ob01 store]$ du -sh *550G clog2.8G ilog96M slog623G sstable[admin@ob03 store]$ du -sh *64M clog #clog目录有768G大幅下降为64M1.1M ilog269M slog623G sstable
3.2 clog文件查看
clog目录只剩余1个文件,文件编号以19497开头[admin@ob02 store]$ ls clog/19497[admin@ob02 clog]$ ls -lrt | grep -v total | head -n1-rw-r--r-- 1 admin admin 67108864 Feb 10 10:25 19497
3.3 磁盘空间使用率查看--使用率降为36%
[admin@ob03 store]$ df -h | grep homeFilesystem Size Used Avail Use% Mounted on/dev/mapper/centos-home 1.8T 631G 1.2T 36% /home

4. 小结
生产环境建议将安装目录、数据目录、日志目录等分开,当clog达到clog_disk_utilization_threshold阈值时,会进行复用/回收。
该参数可以调小,低于目前使用空间,除占用部分IO时,对生产数据无影响。

5. 附--查询clog开始回收的阈值时间
MySQL [oceanbase]> SELECT DATE_FORMAT(gmt_create, '%b%d %H:%i:%s') gmt_create_ , module, event, name1, value1, name2, value2, rs_svr_ip,name3,value3,name4,value4-> FROM __all_rootservice_event_history-> WHERE module IN ('root_service')-> and event='admin_set_config'-> ORDER BY gmt_create DESC-> LIMIT 10;+----------------+--------------+------------------+-------+--------+-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+-------+--------+-------+--------+| gmt_create_ | module | event | name1 | value1 | name2 | value2 | rs_svr_ip | name3 | value3 | name4 | value4 |+----------------+--------------+------------------+-------+--------+-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+-------+--------+-------+--------+| Feb10 09:43:41 | root_service | admin_set_config | ret | 0 | arg | {items:[{name:"clog_disk_utilization_threshold", value:"30", comment:"", zone:"", server:"0.0.0.0", tenant_name:"", exec_tenant_id:1, tenant_ids:[]}], is_inner:false} | 192.168.8.242 | | | | || Feb09 11:15:24 | root_service | admin_set_config | ret | 0 | arg | {items:[{name:"datafile_disk_percentage", value:"35", comment:"", zone:"", server:"0.0.0.0", tenant_name:"", exec_tenant_id:1, tenant_ids:[]}], is_inner:false} | 192.168.8.242 | | | | |+----------------+--------------+------------------+-------+--------+-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+-------+--------+-------+--------+2 rows in set (0.012 sec)
旨在交流,不足之处,还望抛砖。
作者:王坤,微信公众号:rundba,欢迎转载,转载请注明出处。
如需公众号转发,请联系wx: landnow。

文章转载自rundba,如果涉嫌侵权,请发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。




