暂无图片
暂无图片
暂无图片
暂无图片
暂无图片
首页 / MYSQL通用性能优化模板

MYSQL通用性能优化模板

呆呆的私房菜 2024-10-11
255
    Whoami：5年+金融、政府、医疗领域工作经验的DBA
    Certificate：PGCM、OCP、YCP
    Skill：Oracle、Mysql、PostgreSQL、国产数据库
    Platform：CSDN、墨天轮、公众号（呆呆的私房菜）


    阅读本文可以了解关于MYSQL数据库优化的相关手段，包括‍‍‍‍从硬件、操作系统到数据库的优化‍‍‍‍。


    01

    硬件优化

    • CPU：物理机上建议配置更高性能的CPU，不仅是核数越多越好，处理主频也是越高越好。生产环境建议配置不低于8 Cores的CPU；

    • MEM：建议配置更大的物理内存，生产环境建议配置不低于8 GB的物理内存；

    • DISK：建议使用更好的物理I/O设备，配置更高物理IOPS性能的设备，如SSD；

    • NET：如果是构建构建MGR集群，建议使用不低于千兆网络的条件。条件允许的话建议使用万兆网络。


    02

    操作系统优化

    • 1. 采用XFS文件系统，保证在高I/O负载情况下IOPS的性能及稳定性；

      mkfs.xfs -f -L mysql dev/sdb

      • 2. 关闭SWAP交换分区，避免使用SWAP造成数据库性能急剧下降；

        echo "vm.swappiness=0" >> etc/sysctl.conf
        sysctl -p
        swapoff -a

        • 3. 关闭透明大页，避免内存碎片化严重时分配透明大页页面出现较高延迟，从而影响性能；

          echo never > /sys/kernel/mm/transparent_hugepage/enabled
          echo never > /sys/kernle/mm/transparent_hugepage/defrag
          cat /sys/kernle/mm/transparent_hugepage/enabled

          • 4. 调整数据库分区的I/O Scheduler为noop或deadline；

            echo 'noop' > sys/block/sdb/queue/scheduler
            cat /sys/block/sdb/queue/scheduler

            • 5. 确认CPU性能模式为高性能模式；

              cpupower frequency-info --policy
              # 预期输出是：The governor "performanace"，代表高性能模式

              • 6. 确认NUMA模式；

                针对X86架构的CPU，建议关闭NUMA；针对ARM架构的CPU，
                如果单台服务器上运行多个实例，建议开启NUMA提升性能。
                cat etc/default/grub

                • 7. 优化系统内核参数、调整用户资源使用上限；

                  # 调整内核参数
                  echo "fs.file-max = 1000000" >> /etc/sysctl.conf
                  echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
                  echo "net.core.somaxconn = 32768" >> /etc/sysctl.conf
                  echo "net.ipv4.tcp_syncookies = 0" >> /etc/sysctl.conf
                  sysctl -p


                  # 调整用户资源使用上线
                  mysql           soft    nofile         65535
                  mysql           hard    nofile         65535
                  mysql           soft    stack          32768
                  mysql           hard    stack          32768
                  mysql           soft    nproc          65535
                  mysql           hard    nproc          65535

                  • 8. 安装常用系统辅助工具包。

                    yum -y install net-tools perf sysstat iotop tmux strace


                    03

                    数据库优化

                    • 一般情况下，运行MYSQL数据库时可以参考如下my.cnf配置文件就能满足需求了。

                      cat etc/my.cnf
                      # 32C 64G 服务器 mysql配置文件参考
                      [client]
                      port = 3306
                      socket = /mysql/mysql.sock
                      [mysqld]
                      user = mysql
                      port = 3306
                      server_id = 1
                      basedir = /usr
                      datadir = mysql/data
                      socket = /mysql/mysql.sock
                      pid-file = mysql/mysql.pid
                      character-set-server = UTF8MB4
                      skip_name_resolve = 1
                      default_time_zone = "+8:00"
                      bind_address = "0.0.0.0"


                      #performance setttings
                      lock_wait_timeout = 3600
                      open_files_limit = 65535
                      back_log = 1024
                      max_connections = 1024
                      max_connect_errors = 1000000
                      table_open_cache = 2048
                      table_definition_cache = 2048
                      sort_buffer_size = 4M
                      join_buffer_size = 4M
                      read_buffer_size = 8M
                      read_rnd_buffer_size = 4M
                      bulk_insert_buffer_size = 64M
                      thread_cache_size = 768
                      interactive_timeout = 600
                      wait_timeout = 600
                      tmp_table_size = 96M
                      max_heap_table_size = 96M
                      max_allowed_packet = 64M
                      net_buffer_shrink_interval = 180
                      #GIPK
                      loose-sql_generate_invisible_primary_key = ON


                      #log settings
                      log_timestamps = SYSTEM
                      log_error = /data/errlog/error.log
                      log_error_verbosity = 3
                      slow_query_log = 1
                      log_slow_extra = 1
                      slow_query_log_file = /mysql/slowlog/slow.log
                      #设置slow log文件大小1G及总文件数10
                      max_slowlog_size = 1073741824
                      max_slowlog_files = 10
                      long_query_time = 3
                      log_queries_not_using_indexes = 1
                      log_throttle_queries_not_using_indexes = 60
                      min_examined_row_limit = 100
                      log_slow_admin_statements = 1
                      log_slow_replica_statements = 1
                      log_slow_verbosity = FULL
                      log_bin = /data/binlog/binlog
                      binlog_format = ROW
                      sync_binlog = 1
                      binlog_cache_size = 4M
                      max_binlog_cache_size = 6G
                      max_binlog_size = 1G
                      #控制binlog总大小，避免磁盘空间被撑爆
                      binlog_space_limit = 500G
                      binlog_rows_query_log_events = 1
                      binlog_expire_logs_seconds = 604800
                      binlog_checksum = CRC32
                      binlog_order_commits = OFF
                      gtid_mode = ON
                      enforce_gtid_consistency = TRUE


                      #myisam settings
                      key_buffer_size = 32M
                      myisam_sort_buffer_size = 128M


                      #replication settings
                      relay_log_recovery = 1
                      replica_parallel_type = LOGICAL_CLOCK
                      #并行复制线程数可以设置为逻辑CPU数量的2倍
                      replica_parallel_workers = 64
                      binlog_transaction_dependency_tracking = WRITESET
                      replica_preserve_commit_order = 1
                      replica_checkpoint_period = 2


                      #mgr settings
                      loose-plugin_load_add = 'mysql_clone.so'
                      loose-plugin_load_add = 'group_replication.so'
                      loose-group_replication_group_name = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1"
                      #MGR本地节点IP:PORT
                      loose-group_replication_local_address = "192.168.56.11:33061"
                      #MGR集群所有节点IP:PORT
                      loose-group_replication_group_seeds = '192.168.56.11:33061,192.168.56.12:33061,192.168.56.13:33061'
                      loose-group_replication_start_on_boot = ON
                      loose-group_replication_bootstrap_group = OFF
                      loose-group_replication_exit_state_action = READ_ONLY
                      loose-group_replication_flow_control_mode = "QUOTA"
                      loose-group_replication_single_primary_mode = ON
                      loose-group_replication_enforce_update_everywhere_checks = 0
                      loose-group_replication_communication_max_message_size = 10M
                      report_host = 192.168.56.11
                      report_port = 3306


                      #innodb settings
                      innodb_buffer_pool_size = 16G
                      innodb_buffer_pool_instances = 8
                      innodb_data_file_path = ibdata1:12M:autoextend
                      innodb_flush_log_at_trx_commit = 1
                      innodb_log_buffer_size = 32M
                      innodb_log_file_size = 2G
                      innodb_log_files_in_group = 3
                      innodb_redo_log_capacity = 6G
                      innodb_doublewrite_files = 2
                      innodb_max_undo_log_size = 4G
                      # 根据您的服务器IOPS能力适当调整
                      # 一般配普通SSD盘的话，可以调整到 10000 - 20000
                      # 配置高端PCIe SSD卡的话，则可以调整的更高，比如 50000 - 80000
                      innodb_io_capacity = 4000
                      innodb_io_capacity_max = 8000
                      innodb_open_files = 65534
                      # 当需要用CLONE加密特性时，不要选用O_DIRECT模式，否则会比较慢
                      innodb_flush_method = O_DIRECT
                      innodb_lru_scan_depth = 4000
                      innodb_lock_wait_timeout = 10
                      innodb_rollback_on_timeout = 1
                      innodb_print_all_deadlocks = 1
                      innodb_online_alter_log_max_size = 4G
                      innodb_print_ddl_logs = 1
                      innodb_status_file = 1
                      innodb_status_output = 0
                      innodb_status_output_locks = 1
                      innodb_sort_buffer_size = 64M
                      innodb_adaptive_hash_index = 0
                      innodb_numa_interleave = OFF
                      innodb_spin_wait_delay = 20
                      innodb_print_lock_wait_timeout_info = 1
                      #自动杀掉超过5分钟不活跃事务，避免行锁被长时间持有
                      kill_idle_transaction = 300
                      #异步清理大表
                      innodb_data_file_async_purge = ON


                      #innodb monitor settings
                      #根据实际需要开启，会影响数据库性能，但有利于故障诊断和性能优化
                      #innodb_monitor_enable = "module_innodb,module_server,module_dml,module_ddl,module_trx,module_os,module_purge,module_log,module_lock,module_buffer,module_index,module_ibuf_system,module_buffer_page,module_adaptive_hash"
                      #pfs settings
                      performance_schema = 1
                      #performance_schema_instrument = '%memory%=on'
                      performance_schema_instrument = '%lock%=on'


                      本文内容就到这啦，阅读完本篇，相信你对MYSQL通用性能优化手段有了一定的了解了吧！我们下篇再见！


                      点击上方公众号，关注我吧！

                      性能优化mysql集群log文件sql优化mysql
                      文章转载自呆呆的私房菜，如果涉嫌侵权，请发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

                      评论