


各位小伙伴们如果对pg参数调优感兴趣的话,可参考pg社区提供的调优指南:Tuning Your PostgreSQL Server - PostgreSQL wiki
本文我们介绍的是pg调优工具postgresqltuner。
一、安装指南
下载链接:Releases · jfcoz/postgresqltuner · GitHub
依赖环境:yum -y install perl-DBD-pg
二、基本使用
cd opt/postgresqltuner-1.0.1# ./postgresqltuner.pl --host localhost --database postgres --user postgrespostgresqltuner.pl version 1.0.1Checking if OS commands is available on localhost...[OK] OS command OKConnecting to localhost:5432 database postgres with user postgres...[OK] User used for report have super rights===== OS information =====[INFO] OS: linux Version: 3.10.0-957.1.3.el7.x86_64 Arch: x86_64-linux-thread-multi[INFO] OS total memory: 11.73 GB[BAD] Memory overcommitment is allowed on the system. This can lead to OOM Killer killing some PostgreSQL process, which will cause a PostgreSQL server restart (crash recovery)[INFO] sysctl vm.overcommit_ratio=50[BAD] vm.overcommit_ratio is too small, you will not be able to use more than 50*RAM+SWAP for applications[INFO] Running in kvm hypervisor[INFO] Currently used I/O scheduler(s) : deadline[WARN] On virtual machines, sys/block/DISK/queue/rotational is not accurate. Use the --ssd arg if the VM in running on a SSD storage===== General instance informations =====----- Version -----[OK] You are using last 15.1----- Uptime -----[INFO] Service uptime : 26m 32s[WARN] Uptime is less than 1 day. postgresqltuner.pl result may not be accurate----- Databases -----[INFO] Database count (except templates): 1[INFO] Database list (except templates): postgres----- Extensions -----[INFO] Number of activated extensions : 3[INFO] Activated extensions : plpgsql pageinspect pg_freespacemap[WARN] Extensions pg_stat_statements is disabled in database postgres----- Users -----[OK] No user account will expire in less than 7 days[OK] No user with password=username[OK] Password encryption is enabled----- Connection information -----[INFO] max_connections: 100[INFO] current used connections: 6 (6.00%)[INFO] 3 are reserved for super user (3.00%)[INFO] Average connection age : 22m 07s----- Memory usage -----[INFO] configured work_mem: 4.00 MB[INFO] Using an average ratio of work_mem buffers by connection of 150% (use --wmp to change it)[INFO] total work_mem (per connection): 6.00 MB[INFO] shared_buffers: 128.00 MB[INFO] Track activity reserved size : 0.00 B[WARN] maintenance_work_mem is less or equal default value. Increase it to reduce maintenance tasks time[INFO] Max memory usage :shared_buffers (128.00 MB)+ max_connections * work_mem * average_work_mem_buffers_per_connection (100 * 4.00 MB * 150 100 = 600.00 MB)+ autovacuum_max_workers * maintenance_work_mem (3 * 64.00 MB = 192.00 MB)+ track activity size (0.00 B)= 920.00 MB[INFO] effective_cache_size: 4.00 GB[INFO] Size of all databases : 29.68 MB[WARN] shared_buffer is too big for the total databases size, memory is lost[INFO] PostgreSQL maximum memory usage: 7.66% of system RAM[WARN] Max possible memory usage for PostgreSQL is less than 60% of system total RAM. On a dedicated host you can increase PostgreSQL buffers to optimize performances.[INFO] max memory+effective_cache_size is 41.75% of total RAM----- Logs -----[OK] log_hostname is off : no reverse DNS lookup latency[WARN] log of long queries is desactivated. It will be more difficult to optimize query performances[OK] log_statement=none----- Two phase commit -----[OK] Currently no two phase commit transactions----- Autovacuum -----[OK] autovacuum is activated.[INFO] autovacuum_max_workers: 3----- Checkpoint -----[OK] checkpoint_completion_target(0.9) OK----- Disk access -----[OK] fsync is on[OK] synchronize_seqscans is on----- WAL ---------- Planner -----[OK] costs settings are defaults[BAD] some plan features are disabled : enable_partitionwise_aggregate,enable_partitionwise_join===== Database information for database postgres =====----- Database size -----[INFO] Database postgres total size : 15.41 MB[INFO] Database postgres tables size : 8.61 MB (55.88%)[INFO] Database postgres indexes size : 6.80 MB (44.12%)----- Tablespace location -----[OK] No tablespace in PGDATA----- Shared buffer hit rate -----[INFO] shared_buffer_heap_hit_rate: 99.97%[INFO] shared_buffer_toast_hit_rate: 86.88%[INFO] shared_buffer_tidx_hit_rate: 95.06%[INFO] shared_buffer_idx_hit_rate: 99.99%[OK] Shared buffer idx hit rate is very good----- Indexes -----[OK] No invalid indexes[WARN] Some indexes are unused since last statistics: t.t_info_idx----- Procedures -----[WARN] Some user procedures does not have custom cost and rows settings : public.tuple_data_split public.tuple_data_split public.bt_metap public.bt_page_stats public.page_header public.brin_page_type public.brin_metapage_info public.fsm_page_contents public.gin_metapage_info public.gin_page_opaque_info public.hash_page_type public.hash_page_stats public.hash_metapage_info public.heap_tuple_infomask_flags public.gist_page_opaque_info public.get_raw_page public.get_raw_page public.page_checksum public.pg_freespace===== Configuration advices =====----- extension -----[LOW] Enable pg_stat_statements in database postgres to collect statistics on all queries (not only queries longer than log_min_duration_statement in logs)----- index -----[MEDIUM] You have unused indexes in the database since last statistics. Please remove them if they are never use----- proc -----[LOW] You have custom procedures with default cost and rows setting. Please reconfigure them with specific values to help the planer----- report -----[URGENT] Use the --ssd arg if the VM in running on a SSD storage----- sysctl -----[URGENT] set vm.overcommit_memory=2 in /etc/sysctl.conf and run sysctl -p to reload it. This will disable memory overcommitment and avoid postgresql killed by OOM killer.
三、优化内容
1. 操作系统信息检查
① 检查 os version
② 检查 os memory
③ 检查 memory overcommit,若启用,oom时会导致部分pg进程被kill,造成数据库故障。
④ 检查 vm.overcommit_ratio,允许内核分配的内存超过物理内存的百分比
⑤ 检查运行环境是虚拟机还是物理机
⑥ 检查磁盘性能,ssd/机械
2. 实例信息
① 检查数据库版本
② 检查系统启动时间
③ 检查database情况(数量和名称)
④ 检查扩展情况扩展
⑤ 检查用户情况(即将过期用户/用户名密码一致用户/密码加密功能)
⑥ 检查连接情况(最大连接数/当前连接数/超管连接数/平均连接时长)
⑦ 检查内存情况(work_mem/shared_buffer/maintenance_work_mem/effective_cache_size/)
shared_buffers = max_connections * work_mem * average_work_mem_buffer_per_connection + autovacuum_max_workers * maintenance_work_mem + track activity size
⑧ 检查日志情况(慢查询/日志审计)
如果语句运行至少指定的时间量,将导致记录每一个这种完成的语句的持续时间。如果指定值时没有单位,则以毫秒为单位。将这个参数设置为零将打印所有语句的执行时间。设置为 -1 (默认值)将停止记录语句持续时间。例如,如果你设置它为250ms,那么所有运行 250ms 或更久的 SQL 语句将被记录。启用这个参数可以有助于追踪应用中未优化的查询。只有超级用户可以改变这个设置。对于使用扩展查询协议的客户端,解析、绑定和执行步骤的持续时间将被独立记录。当把这个选项和log_statement一起使用时,已经被log_statement记录的语句文本不会在持续时间日志消息中重复。如果你没有使用syslog,我们推荐你使用log_line_prefix记录 PID 或会话 ID,这样你可以使用进程 ID 或会话 ID 把语句消息链接到后来的持续时间消息。
⑨ 检查锁情况
⑩ 检查 autovacuum
指定能同时运行的自动清理进程(除了自动清理启动器之外)的最大数量。默认值为3。该参数只能在服务器启动时设置。
检查 checkpint
指定检查点完成的目标,作为检查点之间总时间的一部分。默认是 0.5。 这个参数只能在postgresql.conf文件中或在服务器命令行上设置。
检查 disk_access(fsync/synchronize_seqscans)
检查 wal
max_wal_size:默认最大1Gmin_wal_size:默认最小80M
检查 planner
enable_partitionwise_aggregate:/** 允许或者禁止查询规划器使用面向分区的分组或聚集,这使得在分区表上的分组或聚集可以在每个分区上分别执行。如果GROUP BY子句不包括分区键,只有部分聚集能够以基于每个分区的方式执行,并且finalization必须最后执行。由于面向分区的分组或聚集在规划期间会使用可观的CPU时间和内存,所以默认值为off。**/enable_partitionwise_join:/**允许或者禁止查询规划器使用面向分区的连接,这使得分区表之间的连接以连接匹配的分区的方式来执行。面向分区的连接当前只适用于连接条件包括所有分区键的情况,连接条件必须是相同的数据类型并且子分区集合要完全匹配。由于面向分区的连接规划在规划期间会使用可观的CPU时间和内存,所以默认值为off。**/
3. postgres数据库信息
① 数据库情况(数据库大小/表大小/索引大小)
② 表空间情况
③ 共享缓冲区命中情况
shared_buffer_heap_hit_rate
shared_buffer_toast_hit_rate
shared_buffer_tidx_hit_rate
shared_buffer_idx_hit_rate
④ 索引情况
⑤ 存储过程




