一个mysql研发测试环境,单机,版本mysql5.7.24。
一线人员发现mysql挂了,磁盘占用100%. 于是他删除了部分binlog,腾出2G空间,启动mysql,正常。
因为不知道哪些数据表的数据可以truncate,他打算回收一些大表delete后的空间。找到一个最大表占14G的,optimize table table_name;发现空间一直上涨,因为会生成临时文件#sql34433.sql。于是想移走几个表腾出空间让optimize完成再移回来(错误操作,这时候没有停止mysql,应用还在读写)。发现mv几个表ibd文件后,空间没有腾出,此时optimize导致磁盘再次占满,mysql中断。
一线人员执行了lsof | grep deleted | awk '{ print $2 }' | xargs -i kill -9 \{\};。回收了刚才mv的空间。重启mysql,无法启动。查看错误日志
2021-10-12 11:25:00 0x2b633c1c6700 InnoDB: Assertion failure in thread 47705210251008 in file btr0pcur.cc line 454
InnoDB: Failing assertion: btr_page_get_prev(next_page, mtr) == btr_pcur_get_block(cursor)->page.id.page_no()
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.