ORA-59327 or ORA-15067 or ORA-59048

在ASM 12.1.0.2和更高的版本中,可能会出现以下报错：

ORA-15067: command or option incompatible with diskgroup redundancy

ORA-59327: Dropping more fail groups is not allowed with disk group redundancy.

当normal冗余度的磁盘组中的 failgroup 小于3，high冗余度的磁盘组中的 failgroup 小于5，是不允许正常删除故障的。

19c
SQL> select GROUP_NUMBER,DISK_NUMBER,FAILGROUP,PATH from v$asm_disk where GROUP_NUMBER=3;

GROUP_NUMBER DISK_NUMBER FAILGROUP                      PATH
------------ ----------- ------------------------------ --------------------------------------------------
           3           0 ARCH_0000                      /dev/asm-arch1
           3           2 ARCH_0002                      /dev/asm-arch3
           3           1 ARCH_0001                      /dev/asm-arch2

SQL> select name,state,type,COMPATIBILITY from v$asm_diskgroup;

NAME                           STATE       TYPE   COMPATIBILITY
------------------------------ ----------- ------ ------------------------------------------------------------
DATA                           MOUNTED     EXTERN 19.0.0.0.0
OCR                            MOUNTED     NORMAL 19.0.0.0.0
ARCH                           MOUNTED     NORMAL 19.0.0.0.0

SQL> select GROUP_NUMBER,DISK_NUMBER,FAILGROUP,PATH from v$asm_disk where GROUP_NUMBER=3;

GROUP_NUMBER DISK_NUMBER FAILGROUP                      PATH
------------ ----------- ------------------------------ --------------------------------------------------
           3           0 ARCH_0000                      /dev/asm-arch1
           3           2 ARCH_0002                      /dev/asm-arch3
           3           1 ARCH_0001                      /dev/asm-arch2

SQL> alter diskgroup arch drop disks in failgroup ARCH_0002;
alter diskgroup arch drop disks in failgroup ARCH_0002
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-59327: Dropping more fail groups is not allowed with disk group redundancy.

SQL>

12.1.0.2 - 18c
SQL> create diskgroup TEST19 normal redundancy disk '/dev/asm0' disk '/dev/asm1' disk '/dev/asm2' attribute 'compatible.asm'='12.2.0.2';

SQL> select name,state,type from v$asm_diskgroup;

NAME         STATE TYPE
---------- ----------- ------
TEST19 MOUNTED NORMAL

SQL> select group_number,substr(path,1,12) Path,substr(name,1,15) Name, substr(failgroup,1,12) Failgroup,header_status,state from v$asm_disk where group_number=(select group_number from v$asm_diskgroup where name='TEST19');

GROUP_NUMBER PATH NAME FAILGROUP HEADER_STATU STATE
------------ ------------ --------------- ------------ ------------ --------
2             /dev/asm2 TEST19_0002 TEST19_0002 MEMBER NORMAL
2             /dev/asm1 TEST19_0001 TEST19_0001 MEMBER NORMAL
2             /dev/asm0 TEST19_0000 TEST19_0000 MEMBER NORMAL

 

SQL> alter diskgroup TEST19 drop disks in failgroup TEST19_0000;

alter diskgroup TEST19 drop disks in failgroup TEST19_0000
*
ERROR at line 1:
ORA-15067: command or option incompatible with diskgroup redundancy         

当然可以使用force 或者是 修改隐含参数来强行删除。

SQL>
SQL>  alter diskgroup arch drop disks in failgroup ARCH_0002 force;

Diskgroup altered.

SQL> select GROUP_NUMBER,DISK_NUMBER,FAILGROUP,PATH from v$asm_disk where GROUP_NUMBER=3;

GROUP_NUMBER DISK_NUMBER FAILGROUP                      PATH
------------ ----------- ------------------------------ --------------------------------------------------
           3           2 ARCH_0002
           3           0 ARCH_0000                      /dev/asm-arch1
           3           1 ARCH_0001                      /dev/asm-arch2


SQL> alter system set "_asm_disable_failgroup_count_checking"=true sid='*';

System altered.

SQL> show parameter _asm_disable_failgroup_count_checking

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
_asm_disable_failgroup_count_checkin boolean     TRUE
g
SQL> alter diskgroup arch drop disks in failgroup ARCH_0002;

Diskgroup altered.

SQL> select GROUP_NUMBER,DISK_NUMBER,name,FAILGROUP,PATH from v$asm_disk where GROUP_NUMBER=3;

SQL> select GROUP_NUMBER,DISK_NUMBER,name,FAILGROUP,PATH,state from v$asm_disk where GROUP_NUMBER=3;

GROUP_NUMBER DISK_NUMBER NAME                           FAILGROUP                      PATH                                               STATE
------------ ----------- ------------------------------ ------------------------------ -------------------------------------------------- --------
           3           3 _DROPPED_0003_ARCH             ARCH_0002                                                                         FORCING
           3           0 ARCH_0000                      ARCH_0000                      /dev/asm-arch1                                     NORMAL
           3           1 ARCH_0001                      ARCH_0001                      /dev/asm-arch2                                     NORMAL

但这都会出现后遗症。

虽然提示，故障组删除成功了，但是仔细观察ARCH_0002故障组的磁盘还属于group 3 且在ASM的告警日志中提示

NOTE: grp 3 disk 3 not expelled because ASM metadata redundancy has not been restored due to an insufficientnumber of failure groups

disk 3 没有被驱逐(被删除的盘)。因为ASM元数据冗余由于故障组数量不足而没有恢复。

在19c的高版本中，查询v$asm_operation会发现REBALANCE失败，19.3的基础版本没有出现该问题

GROUP_NUMBER OPERA PASS      STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE                            CON_ID
------------ ----- --------- ---- ---------- ---------- ---------- ---------- ---------- ----------- -------------------------------------------- ----------
           1 REBAL COMPACT   WAIT          1                                                                                            0
           1 REBAL REBALANCE ERRS          1                                                         ORA-59048                          0
           1 REBAL REBUILD   WAIT          1                                                                                            0
           1 REBAL RESYNC    WAIT          1                                                                                            0

解决方案

1)Add more Failgroups to the Normal/High Redundancy Diskgroup.

       SQL> alter diskgroup <DGNAME> add failgroup <FAILGROUP_NAME> disk '<path>','<path>'.....;

                        = OR =

2)Add Quorum Failgroups to the Normal/High Redundancy Diskgroup.  

       SQL>alter diskgroup <dgname> add quorum failgroup <Quorum_Failgroup_name> disk '<NFS path>';

 

   Highly recommended for Exadata environments having High Redundancy Diskgroup with only 3 Cell Servers and 2 Database Nodes.

   Add quorum disks in Exadata:
        /opt/oracle.SupportTools/quorumdiskmgr --create --config --owner=<GI_owner> --group=<GI_Group> --network-iface-list="ibx,ibx"
        /opt/oracle.SupportTools/quorumdiskmgr --create --target --asm-disk-group=<dgname> --visible-to="<compute Node IP's>"
        /opt/oracle.SupportTools/quorumdiskmgr --create --device --target-ip-list="<compute Node IP's>"

 

   Note: Quorum Disk Manager utility is introduced in Oracle Exadata release 12.1.2.3.0.

             Please refer Engineered Systems documentation for more details on the quorumdiskmgr configuration steps and syntax.

https://docs.oracle.com/en/engineered-systems/exadata-database-machine/dbmmn/maintaining-exadata-database-servers.html#GUID-6F322D77-F1DF-41A3-B807-131A9ABD4FBC

                        = OR  =

3)Recreate the Normal/High Redundancy Diskgroups with requisite number of Failgroups ( Normal Redundancy -3 Failgroups , High Redundancy - 5 Failgroups).

       SQL> create Diskgroup <DGNAME> normal redundancy
                failgroup <FAILGROUP1> disk '<path>','<path>'...
                failgroup <FAILGROUP2> disk '<path>','<path>'....
                failgroup <FAILGROUP2> disk '<path>','<path>'...;

Then copy files or RMAN restore from existing Diskgroup to this newly created Diskgroup.

公众号：ZXM的学习笔记
墨天伦：https://www.modb.pro/u/15528