Oracle 数据库异常关闭,共享内存未释放,导致无法启动。
以下内容为测试环境复原故障现场:
流程:
尝试启动数据库出现报错信息:
[oracle@server1 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Thu Jan 21 00:18:04 2021
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup;
ORA-00845: MEMORY_TARGET not supported on this system
警告日志:
Starting ORACLE instance (normal)
WARNING: You are trying to use the MEMORY_TARGET feature.
This feature requires the /dev/shm file system to be mounted for at least 834666496 bytes.
/dev/shm is either not mounted or is mounted with available space less than this size.
Please fix this so that MEMORY_TARGET can work as expected.
Current available is 804745216 and used is 238903296 bytes.
Ensure that the mount point is /dev/shm for this directory.
memory_target needs larger /dev/shm
排查发现Oracle共享内存段未释放:
[oracle@server1 ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
50G 15G 33G 32% /
tmpfs 996M 228M 768M 23% /dev/shm
/dev/sda1 477M 40M 412M 9% /boot
/dev/mapper/VolGroup-lv_home
144G 60M 136G 1% /home
[oracle@server1 ~]$ ipcs
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 2359299 oracle 640 4096 0
0xba1697c4 2424837 oracle 640 4096 0
------ Semaphore Arrays --------
key semid owner perms nsems
0x4d96aff0 753666 oracle 640 154
------ Message Queues --------
key msqid owner perms used-bytes messages
使用ipcrm命令进行手工释放共享内存段:
[oracle@server1 ~]$ ipcrm -m 2359299
[oracle@server1 ~]$ ipcrm -m 2424837
[oracle@server1 ~]$ ipcrm -s 753666
[oracle@server1 ~]$ ipcs
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
------ Semaphore Arrays --------
key semid owner perms nsems
------ Message Queues --------
key msqid owner perms used-bytes messages
伴随着如上操作:
警告日志中出现如下内容:
Thu Jan 21 00:22:30 2021
Thu Jan 21 00:22:30 2021
Errors in file /oracle/app/diag/rdbms/prod/prod/trace/prod_gen0_3022.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Errors in file /oracle/app/diag/rdbms/prod/prod/trace/prod_mman_3030.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Thu Jan 21 00:22:30 2021
Errors in file /oracle/app/diag/rdbms/prod/prod/trace/prod_psp0_3016.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Thu Jan 21 00:22:30 2021
Errors in file /oracle/app/diag/rdbms/prod/prod/trace/prod_smon_3038.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Thu Jan 21 00:22:30 2021
Errors in file /oracle/app/diag/rdbms/prod/prod/trace/prod_dbrm_3026.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Thu Jan 21 00:22:30 2021
Errors in file /oracle/app/diag/rdbms/prod/prod/trace/prod_q000_3116.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
MMAN (ospid: 3030): terminating the instance due to error 27157
Instance terminated by MMAN, pid = 3030
Errors in file /oracle/app/diag/rdbms/prod/prod/trace/prod_mman_3030.trc:
ORA-27300: OS system dependent operation:semctl failed with status: 22
ORA-27301: OS failure message: Invalid argument
ORA-27302: failure occurred at: sskgpwrm1
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Thu Jan 21 00:22:31 2021
Instance termination got error 27120 from SGA destruction.
Error cleared. Process exiting.
尝试再次启动:
SQL> startup;
ORA-00845: MEMORY_TARGET not supported on this system
依旧不行
进而查看/dev/shm空间是否释放:
[oracle@server1 shm]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
50G 15G 33G 32% /
tmpfs 996M 228M 768M 23% /dev/shm
/dev/sda1 477M 40M 412M 9% /boot
/dev/mapper/VolGroup-lv_home
144G 60M 136G 1% /home
进入/dev/shm一探究竟:
[oracle@server1 shm]$ ls
ora_prod_2359299_0 ora_prod_2392068_119 ora_prod_2392068_142 ora_prod_2392068_166 ora_prod_2392068_19 ora_prod_2392068_37 ora_prod_2392068_60 ora_prod_2392068_84
ora_prod_2359299_1 ora_prod_2392068_12 ora_prod_2392068_143 ora_prod_2392068_167 ora_prod_2392068_190 ora_prod_2392068_38 ora_prod_2392068_61 ora_prod_2392068_85
ora_prod_2359299_2 ora_prod_2392068_120 ora_prod_2392068_144 ora_prod_2392068_168 ora_prod_2392068_191 ora_prod_2392068_39 ora_prod_2392068_62 ora_prod_2392068_86
ora_prod_2392068_0 ora_prod_2392068_121 ora_prod_2392068_145 ora_prod_2392068_169 ora_prod_2392068_192 ora_prod_2392068_4 ora_prod_2392068_63 ora_prod_2392068_87
ora_prod_2392068_1 ora_prod_2392068_122 ora_prod_2392068_146 ora_prod_2392068_17 ora_prod_2392068_193 ora_prod_2392068_40 ora_prod_2392068_64 ora_prod_2392068_88
ora_prod_2392068_10 ora_prod_2392068_123 ora_prod_2392068_147 ora_prod_2392068_170 ora_prod_2392068_194 ora_prod_2392068_41 ora_prod_2392068_65 ora_prod_2392068_89
ora_prod_2392068_100 ora_prod_2392068_124 ora_prod_2392068_148 ora_prod_2392068_171 ora_prod_2392068_195 ora_prod_2392068_42 ora_prod_2392068_66 ora_prod_2392068_9
ora_prod_2392068_101 ora_prod_2392068_125 ora_prod_2392068_149 ora_prod_2392068_172 ora_prod_2392068_2 ora_prod_2392068_43 ora_prod_2392068_67 ora_prod_2392068_90
ora_prod_2392068_102 ora_prod_2392068_126 ora_prod_2392068_15 ora_prod_2392068_173 ora_prod_2392068_20 ora_prod_2392068_44 ora_prod_2392068_68 ora_prod_2392068_91
ora_prod_2392068_103 ora_prod_2392068_127 ora_prod_2392068_150 ora_prod_2392068_174 ora_prod_2392068_21 ora_prod_2392068_45 ora_prod_2392068_69 ora_prod_2392068_92
ora_prod_2392068_104 ora_prod_2392068_128 ora_prod_2392068_151 ora_prod_2392068_175 ora_prod_2392068_22 ora_prod_2392068_46 ora_prod_2392068_7 ora_prod_2392068_93
ora_prod_2392068_105 ora_prod_2392068_129 ora_prod_2392068_152 ora_prod_2392068_176 ora_prod_2392068_23 ora_prod_2392068_47 ora_prod_2392068_70 ora_prod_2392068_94
ora_prod_2392068_106 ora_prod_2392068_13 ora_prod_2392068_153 ora_prod_2392068_177 ora_prod_2392068_24 ora_prod_2392068_48 ora_prod_2392068_71 ora_prod_2392068_95
ora_prod_2392068_107 ora_prod_2392068_130 ora_prod_2392068_154 ora_prod_2392068_178 ora_prod_2392068_25 ora_prod_2392068_49 ora_prod_2392068_72 ora_prod_2392068_96
ora_prod_2392068_108 ora_prod_2392068_131 ora_prod_2392068_155 ora_prod_2392068_179 ora_prod_2392068_26 ora_prod_2392068_5 ora_prod_2392068_73 ora_prod_2392068_97
ora_prod_2392068_109 ora_prod_2392068_132 ora_prod_2392068_156 ora_prod_2392068_18 ora_prod_2392068_27 ora_prod_2392068_50 ora_prod_2392068_74 ora_prod_2392068_98
ora_prod_2392068_11 ora_prod_2392068_133 ora_prod_2392068_157 ora_prod_2392068_180 ora_prod_2392068_28 ora_prod_2392068_51 ora_prod_2392068_75 ora_prod_2392068_99
ora_prod_2392068_110 ora_prod_2392068_134 ora_prod_2392068_158 ora_prod_2392068_181 ora_prod_2392068_29 ora_prod_2392068_52 ora_prod_2392068_76 ora_prod_2424837_0
ora_prod_2392068_111 ora_prod_2392068_135 ora_prod_2392068_159 ora_prod_2392068_182 ora_prod_2392068_3 ora_prod_2392068_53 ora_prod_2392068_77 pulse-shm-3740554943
ora_prod_2392068_112 ora_prod_2392068_136 ora_prod_2392068_16 ora_prod_2392068_183 ora_prod_2392068_30 ora_prod_2392068_54 ora_prod_2392068_78 pulse-shm-613326773
ora_prod_2392068_113 ora_prod_2392068_137 ora_prod_2392068_160 ora_prod_2392068_184 ora_prod_2392068_31 ora_prod_2392068_55 ora_prod_2392068_79
ora_prod_2392068_114 ora_prod_2392068_138 ora_prod_2392068_161 ora_prod_2392068_185 ora_prod_2392068_32 ora_prod_2392068_56 ora_prod_2392068_8
ora_prod_2392068_115 ora_prod_2392068_139 ora_prod_2392068_162 ora_prod_2392068_186 ora_prod_2392068_33 ora_prod_2392068_57 ora_prod_2392068_80
ora_prod_2392068_116 ora_prod_2392068_14 ora_prod_2392068_163 ora_prod_2392068_187 ora_prod_2392068_34 ora_prod_2392068_58 ora_prod_2392068_81
ora_prod_2392068_117 ora_prod_2392068_140 ora_prod_2392068_164 ora_prod_2392068_188 ora_prod_2392068_35 ora_prod_2392068_59 ora_prod_2392068_82
ora_prod_2392068_118 ora_prod_2392068_141 ora_prod_2392068_165 ora_prod_2392068_189 ora_prod_2392068_36 ora_prod_2392068_6 ora_prod_2392068_83
使用fuser命令判断是否有进程占用:
[oracle@server1 shm]$ fuser ora_prod_2359299_0
进行手动清理:
[oracle@server1 shm]$ rm -rf ora_prod_*
[oracle@server1 shm]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
50G 15G 33G 32% /
tmpfs 996M 72K 996M 1% /dev/shm
/dev/sda1 477M 40M 412M 9% /boot
/dev/mapper/VolGroup-lv_home
144G 60M 136G 1% /home
再次尝试启动:
SQL> startup;
ORACLE instance started.
Total System Global Area 830930944 bytes
Fixed Size 2257800 bytes
Variable Size 524291192 bytes
Database Buffers 297795584 bytes
Redo Buffers 6586368 bytes
Database mounted.
Database opened.
正常~
20210120 记录!
「喜欢这篇文章，您的关注和赞赏是给作者最好的鼓励」
关注作者
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文章的来源（墨天轮），文章链接，文章作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。
