添加和删除相关操作votedisk管理

1，磁盘状态查询
col PATH format a20;
set linesize 1000;
select GROUP_NUMBER,DISK_NUMBER,NAME,PATH,HEADER_STATUS,MOUNT_STATUS,FAILGROUP,VOTING_FILE,FAILGROUP_TYPE,STATE from v$asm_disk order by 1,2;
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU MOUNT_S FAILGROUP V FAILGRO STATE
------------ ----------- ------------------------------ -------------------- ------------ ------- ------------------------------ - ------- --------
0 0 /nfs/rhvot03 FORMER CLOSED N REGULAR NORMAL
0 1 /nfs/rhvot01 FORMER CLOSED N REGULAR NORMAL
0 2 /nfs/rhvot02 FORMER CLOSED N REGULAR NORMAL
0 3 /nfs/rhvot04 FORMER CLOSED N REGULAR NORMAL
2 1 SYS_0001 /dev/sdc MEMBER CACHED G02 Y REGULAR NORMAL
2 2 SYS_0002 /dev/sdd MEMBER CACHED G03 Y REGULAR NORMAL
2 3 SYS_0003 /dev/sdb MEMBER CACHED VOT002 Y QUORUM NORMAL
7 rows selected.
2，删除QUORUM磁盘测试
SQL>
SQL> alter diskgroup sys drop QUORUM disk 'SYS_0003' ; <---------删除失败
alter diskgroup sys drop QUORUM disk 'SYS_0003'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-59327: Dropping more fail groups is not allowed with disk group redundancy.
##测试添加一个新的 QUORUM FAILGROUP vot001 在此删除SYS_0003;
SQL> alter diskgroup SYS add QUORUM failgroup VOT001 disk '/nfs/rhvot01';
Diskgroup altered.
SQL> @c
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU MOUNT_S FAILGROUP V FAILGRO STATE
------------ ----------- ------------------------------ -------------------- ------------ ------- ------------------------------ - ------- --------
0 0 /nfs/rhvot03 FORMER CLOSED N REGULAR NORMAL
0 1 /nfs/rhvot02 FORMER CLOSED N REGULAR NORMAL
0 2 /nfs/rhvot04 FORMER CLOSED N REGULAR NORMAL
2 0 SYS_0000 /nfs/rhvot01 MEMBER CACHED VOT001 Y QUORUM NORMAL <---新添加vot001 磁盘组。
2 1 SYS_0001 /dev/sdc MEMBER CACHED G02 Y REGULAR NORMAL
2 2 SYS_0002 /dev/sdd MEMBER CACHED G03 Y REGULAR NORMAL
2 3 SYS_0003 /dev/sdb MEMBER CACHED VOT002 N QUORUM NORMAL
7 rows selected.
SQL> alter diskgroup sys drop QUORUM disk 'SYS_0003' ; <-----正常删除，votedisk 替换完成。
Diskgroup altered.
SQL> @c
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU MOUNT_S FAILGROUP V FAILGRO STATE
------------ ----------- ------------------------------ -------------------- ------------ ------- ------------------------------ - ------- --------
0 0 /nfs/rhvot03 FORMER CLOSED N REGULAR NORMAL
0 1 /nfs/rhvot02 FORMER CLOSED N REGULAR NORMAL
0 2 /dev/sdb FORMER CLOSED N REGULAR NORMAL
0 3 /nfs/rhvot04 FORMER CLOSED N REGULAR NORMAL
2 0 SYS_0000 /nfs/rhvot01 MEMBER CACHED VOT001 Y QUORUM NORMAL
2 1 SYS_0001 /dev/sdc MEMBER CACHED G02 Y REGULAR NORMAL
2 2 SYS_0002 /dev/sdd MEMBER CACHED G03 Y REGULAR NORMAL
7 rows selected
[grid@ora03 ~]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 4b0621fe36814ff3bf126178504a8188 (/dev/sdc) [SYS]
2. ONLINE b82f27d06c614f44bf4fcc38212e6f23 (/dev/sdd) [SYS]
3. ONLINE 3dc407fb48fe4f26bfeef2bfd964d0f8 (/nfs/rhvot01) [SYS]
3,模拟破坏启动一个votedisk
[grid@ora03 ~]$ dd if=/dev/zero of=/nfs/rhvot01 bs=512M count=2
2+0 records in
2+0 records out
1073741824 bytes (1.1 GB) copied, 38.9174 s, 27.6 MB/s
[grid@ora03 ~]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 4b0621fe36814ff3bf126178504a8188 (/dev/sdc) [SYS]
2. ONLINE b82f27d06c614f44bf4fcc38212e6f23 (/dev/sdd) [SYS]
3. PENDOFFL 3dc407fb48fe4f26bfeef2bfd964d0f8 (/nfs/rhvot01) [SYS] <=====状态显示异常。
Located 3 voting disk(s).
@破坏一块votdisk 整个集群显示正常。
[grid@ora03 ~]$ crsctl check cluster
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
4,破坏votedisk,修复
@丢失一个votedisk。asm磁盘组状态显示正常，votedisk 显示异常。
@查询磁盘组状态：
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 0 VOT_0000 /nfs/rhvot01 MEMBER VOT_0000 N QUORUM NORMAL CACHED ONLINE 0
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 N REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 N REGULAR NORMAL CACHED ONLINE 0
# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE b1aaa5d6505c4ff1bf29569ee5a290df (/nfs/rhvot02) [VOT]
2. ONLINE eace3d7bd3164f05bf25c3967afba774 (/nfs/rhvot03) [VOT]
3. ONLINE 02abdd0221444fd7bfc3b61c30dedd92 (/nfs/rhvot01) [VOT] <====状态正常
#模拟破坏/nfs/rhvot01磁盘：（QUORUM磁盘组磁盘）
[grid@ora03 ~]$ dd if=/dev/zero of=/nfs/rhvot01 bs=512M count=2
2+0 records in
2+0 records out
1073741824 bytes (1.1 GB) copied, 33.9351 s, 31.6 MB/s
[grid@ora03 ~]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE b1aaa5d6505c4ff1bf29569ee5a290df (/nfs/rhvot02) [VOT]
2. ONLINE eace3d7bd3164f05bf25c3967afba774 (/nfs/rhvot03) [VOT]
3. PENDOFFL 02abdd0221444fd7bfc3b61c30dedd92 (/nfs/rhvot01) [VOT] <====状态异常
Located 3 voting disk(s).
[grid@ora03 ~]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE b1aaa5d6505c4ff1bf29569ee5a290df (/nfs/rhvot02) [VOT]
2. ONLINE eace3d7bd3164f05bf25c3967afba774 (/nfs/rhvot03) [VOT]
3. OFFLINE 02abdd0221444fd7bfc3b61c30dedd92 (/nfs/rhvot01) [VOT] <====状态异常
#asm 磁盘状态仍显示正常。
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 0 VOT_0000 /nfs/rhvot01 MEMBER VOT_0000 Y QUORUM NORMAL CACHED ONLINE 0 <====故障磁盘 。
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 Y REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 Y REGULAR NORMAL CACHED ONLINE 0
SQL>
4.1 修复方法一：
@手动离线故障磁盘，重新添加QUORUM磁盘。
#offline故障磁盘：
SQL> alter diskgroup vot offline QUORUM disk 'VOT_0000';
Diskgroup altered.
SQL> @c
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 0 VOT_0000 UNKNOWN VOT_0000 N QUORUM NORMAL MISSING OFFLINE 300
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 Y REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 Y REGULAR NORMAL CACHED ONLINE 0
#重新添加QUORUM磁盘：
SQL> alter diskgroup vot add QUORUM disk '/nfs/rhvot01';
Diskgroup altered.
SQL> @c
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 0 VOT_0000 UNKNOWN VOT_0000 N QUORUM NORMAL MISSING OFFLINE 300
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 Y REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 Y REGULAR NORMAL CACHED ONLINE 0
1 3 VOT_0003 /nfs/rhvot01 MEMBER VOT_0003 Y QUORUM NORMAL CACHED ONLINE 0
SQL>
#检测votedisk状态：
[grid@ora03 ~]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE b1aaa5d6505c4ff1bf29569ee5a290df (/nfs/rhvot02) [VOT]
2. ONLINE eace3d7bd3164f05bf25c3967afba774 (/nfs/rhvot03) [VOT]
3. ONLINE 60cb21e765f94f52bf4750094a42a5a5 (/nfs/rhvot01) [VOT]
Located 3 voting disk(s).
#VOT_0000 可以手动删除可以，自动删除；
#手动删除命令：
alter diskgroup vot drop QUORUM disk 'VOT_0000';
#自动删除信息：
col name format a30
select name,value from v$asm_attribute where group_number=1 and name like '%repair_time%';
NAME VALUE
------------------------------ ----------
disk_repair_time 5m
failgroup_repair_time 5m
@自动删除故障日盘之日信息：
2023-08-22T13:19:22.939881+08:00
WARNING: Disk 0 (VOT_0000) in group 1 will be dropped in: (116) secs on ASM inst 1
....
....
WARNING: Disk 0 (VOT_0000) in group 1 will be dropped in: (0) secs on ASM inst 1
WARNING: PST-initiated drop of 1 disk(s) in group 1(.1392552253))
2023-08-22T13:22:27.239957+08:00
SQL> alter diskgroup VOT drop quorum disk VOT_0000 force /* ASM SERVER */
2023-08-22T13:22:27.246626+08:00
NOTE: cache closing disk 0 of grp 1: (not open) VOT_0000
NOTE: GroupBlock outside rolling migration privileged region
NOTE: requesting all-instance membership refresh for group=1
2023-08-22T13:22:27.381025+08:00
GMON updating for reconfiguration, group 1 at 83 for pid 35, osid 36045
2023-08-22T13:22:27.383435+08:00
NOTE: cache closing disk 0 of grp 1: (not open) VOT_0000
2023-08-22T13:22:27.403405+08:00
NOTE: group 1 PST updated.
2023-08-22T13:22:27.428865+08:00
NOTE: cache closing disk 0 of grp 1: (not open) VOT_0000
2023-08-22T13:22:27.447720+08:00
NOTE: membership refresh pending for group 1/0x5300a93d (VOT)
NOTE: Attempting voting file refresh on diskgroup VOT
NOTE: Refresh completed on diskgroup VOT. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup VOT
NOTE: Attempting voting file relocation on diskgroup VOT
NOTE: Successful voting file relocation on diskgroup VOT
2023-08-22T13:22:28.219174+08:00
NOTE: cache closing disk 0 of grp 1: (not open) _DROPPED_0000_VOT
2023-08-22T13:22:28.222827+08:00
GMON querying group 1 at 84 for pid 26, osid 4471
2023-08-22T13:22:28.226638+08:00
NOTE: cache closing disk 0 of grp 1: (not open) _DROPPED_0000_VOT
2023-08-22T13:22:28.235135+08:00
SUCCESS: refreshed membership for 1/0x5300a93d (VOT)
2023-08-22T13:22:28.236838+08:00
SUCCESS: alter diskgroup VOT drop quorum disk VOT_0000 force /* ASM SERVER */
@自动删除故障时间REPAIR_TIMER显示：
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 0 VOT_0000 UNKNOWN VOT_0000 N QUORUM NORMAL MISSING OFFLINE 300 <===在300s 以后将被drop;
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 Y REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 Y REGULAR NORMAL CACHED ONLINE 0
1 3 VOT_0003 /nfs/rhvot01 MEMBER VOT_0003 Y QUORUM NORMAL CACHED ONLINE 0
SQL> @c
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 0 VOT_0000 UNKNOWN VOT_0000 N QUORUM NORMAL MISSING OFFLINE 116 <===在116s 以后将被drop;
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 Y REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 Y REGULAR NORMAL CACHED ONLINE 0
1 3 VOT_0003 /nfs/rhvot01 MEMBER VOT_0003 Y QUORUM NORMAL CACHED ONLINE 0
4.2 修复方法二：（不建议）
@添加新磁盘，删除故障磁盘。
SQL> alter diskgroup vot add QUORUM disk '/nfs/rhvot01'; <========使用原来的磁盘，失败，需要指定新磁盘。
alter diskgroup vot add QUORUM disk '/nfs/rhvot01'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15029: disk '/nfs/rhvot01' is already mounted by this instance
SQL> @c
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 Y REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 Y REGULAR NORMAL CACHED ONLINE 0
1 3 VOT_0003 /nfs/rhvot01 MEMBER VOT_0003 Y QUORUM NORMAL CACHED ONLINE 0
SQL> alter diskgroup vot add QUORUM disk '/nfs/rhvot04'; <========添加新磁盘
Diskgroup altered.
SQL> @c
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 0 VOT_0000 /nfs/rhvot04 MEMBER VOT_0000 Y QUORUM NORMAL CACHED ONLINE 0
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 Y REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 Y REGULAR NORMAL CACHED ONLINE 0
1 3 VOT_0003 /nfs/rhvot01 CANDIDATE VOT_0003 Y QUORUM NORMAL CACHED ONLINE 0
SQL> alter diskgroup vot drop QUORUM disk 'VOT_0003'; <====删除故障磁盘，失败，一直显示DROPPING。
Diskgroup altered.
SQL> @c
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 0 VOT_0000 /nfs/rhvot04 MEMBER VOT_0000 Y QUORUM NORMAL CACHED ONLINE 0
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 Y REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 Y REGULAR NORMAL CACHED ONLINE 0
1 3 VOT_0003 /nfs/rhvot01 CANDIDATE VOT_0003 Y QUORUM DROPPING CACHED ONLINE 0 <===状态DROPPING。
SQL> alter diskgroup vot drop QUORUM disk 'VOT_0003' force; <==========需要指定force 才可以强制删除。
Diskgroup altered.
SQL> @c
GROUP_NUMBER DISK_NUMBER NAME PATH HEADER_STATU FAILGROUP V FAILGRO STATE MOUNT_S MODE_ST REPAIR_TIMER
------------ ----------- ------------------------------ -------------------- ------------ ------------------------------ - ------- -------- ------- ------- ------------
1 0 VOT_0000 /nfs/rhvot04 MEMBER VOT_0000 Y QUORUM NORMAL CACHED ONLINE 0
1 1 VOT_0001 /nfs/rhvot02 MEMBER VOT_0001 Y REGULAR NORMAL CACHED ONLINE 0
1 2 VOT_0002 /nfs/rhvot03 MEMBER VOT_0002 Y REGULAR NORMAL CACHED ONLINE 0
SQL>
#检测votdisk 状态：
[grid@ora03 ~]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE b1aaa5d6505c4ff1bf29569ee5a290df (/nfs/rhvot02) [VOT]
2. ONLINE eace3d7bd3164f05bf25c3967afba774 (/nfs/rhvot03) [VOT]
3. ONLINE 0602bd2d6efc4f2bbf95ed20a1a896a6 (/nfs/rhvot04) [VOT]
Located 3 voting disk(s).
#查询命令参考：
col PATH format a20;
set linesize 1000;
select GROUP_NUMBER,DISK_NUMBER,NAME,PATH,HEADER_STATUS,FAILGROUP,VOTING_FILE,FAILGROUP_TYPE,STATE,mount_status,mode_status,REPAIR_TIMER from v$asm_disk where name not like '%SYS%' order by 1,2;
Oracle ASM快速镜像重新同步
要使用此功能，磁盘组兼容性属性必须设置为11.1或更高
failgroup_repair_time：此属性指定在启动放置操作之前修复磁盘并使其恢复联机的时间间隔。此属性只能在更改磁盘组时设置，并且仅适用于普通和高冗余磁盘组
disk_repair_time：此属性指定磁盘组中故障组的默认修复时间。如果Oracle ASM确定整个故障组已失败，则使用故障组修复时间。默认值为24小时（ 24h ）
col name format a30
select name,value from v$asm_attribute where group_number=1 and name like '%repair_time%';