Applies to:
Oracle Database - Enterprise Edition - Version 11.2.0.0 and later
Oracle Database Cloud Schema Service - Version N/A and later
Oracle Database Exadata Express Cloud Service - Version N/A and later
Oracle Database Exadata Cloud Machine - Version N/A and later
Oracle Cloud Infrastructure - Database Service - Version N/A and later
Information in this document applies to any platform.
Symptoms
RAC/standalone configuration with NORMAL redundancy diskgroups.
ASM disks configured with ASMLIB.
1) Following storage operations ASM disks cannot be onlined/added back due to ORA-15186 / ORA-15282: ASM disk is not visible cluster-wide.
SQL> alter diskgroup ELETMP add failgroup ELETMP_FFG disk 'ORCL:SE9990V00A1P1' rebalance power 11
ORA-15186: ASMLIB error function = [asm_open(global)], error = [5], mesg = [No such device]
ORA-15025: could not open disk "ORCL:SE9990V00A1P1"
2) v$asm_disk shows disks with status CLOSED/UNKNOWN
==================================================================================
0 1 CLOSED UNKNOWN ASM Library - version 2.0.4 (KABI_V2) SE9990V00A1P1 ORCL:SE9990V00A1P1
0 0 CLOSED UNKNOWN ASM Library - version 2.0.4 (KABI_V2) SE9990V0093P1 ORCL:SE9990F0093P1
3) alert_+ASM.log shows:
ORA-15025: could not open disk "ORCL:SE9990V00A1P1"
and on the other nodes
NOTE: disk validation pending for group 1/0xf13a9d5b (ELETMP)
ORA-15186: ASMLIB error function = [asm_open(global)], error = [5], mesg = [No such device]
4) "oracleasm scandisks" has been run and correctly reinstantiated ASMLIB disks
Disk "SE9990V0093P1" is a valid ASM disk on device [253,30]
Disk "SE9990V00A1P1" is a valid ASM disk on device [253,48]
5) KFED reports a correct disk header (MEMBER) on all nodes
kfdhdb.driver.provstr: ORCLDISKSE9990V00A1P1 ; 0x000: length=21
kfdhdb.dsknum: 0 ; 0x024: 0x0000
kfdhdb.grptyp: 2 ; 0x026: KFDGTP_NORMAL
kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER>>>>>
kfdhdb.dskname: SE9990V00A1P1 ; 0x028: length=13
kfdhdb.grpname: ELEVENDG ; 0x048: length=8
kfdhdb.fgname: ELEVENDG_FFG ; 0x068: length=12
6) KFOD discovers all of them on all nodes (either as ORCL or /dev/oracleasm/disks)
================================================================================
...
13: 51199 Mb ORCL:SE9990V0093P1 <unknown> <unknown>
14: 20473 Mb ORCL:SE9990V00A1P1 <unknown> <unknown>
================================================================================
7) multipathing configuration is fine on all nodes (either "multipath -ll" or "blkid" )
Changes
Storage maintenance (like offlining all paths to disks) or any kind of storage operations (like replication) which implies a remaping of /dev/mapper/* devices and a change in (major,minor)
Cause
Bug 17047653 - ORA-15186: ASMLIB ERROR FUNCTION ERROR = [5]/DISKS SEEN CLOSED/UNKNOWN
closed as duplicate of
Bug 17896002 - AFTER RUNNING RENAMEDISK DB COMMAND IS FAILING STATING ASMLIB HEADER IS UNKNOWN
Solution
1) Apply the patch 17896002
2) Workaround: restart ASM instances (CRS stack)
