存储故障，修复后
Arch_dg无法mount
尝试手动启动时候报错
SQL> alter diskgroup NHCPDB_ARCH mount;
alter diskgroup NHCPDB_ARCH mount
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15096: lost disk write detected
开sr，oracle 也未能解决，是磁盘故障，
解决办法如下，
- Startup nomount
- 将参数中存在于arch dg的控制文档拿掉
alter system set control_files='xxx' scope=spfile;
- 在本机查找最近的控制文件备份记录
Netbackup/bin/ ./bplist –C nhcprac1 –t 4 –l –R /
2. 恢复控制文件并启动到mount
RUN {
allocate channel D1 type 'sbt_tape' parms 'SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.so64';
send 'NB_ORA_SERV=nbuserver,NB_ORA_CLIENT=nhcprac1';
restore controlfile from '/cntrl_6308_1_1031792559';---最新的控制文件备份
RELEASE CHANNEL D1;
}
3. 恢复成功后，启动到mount，将涉及到archdg 路径的参数都改道data dg
alter system set log_archive_dest_1="Location=+NHCPDB_DATA" scope=spfile;
alter system set db_recovery_file_dest='' scope=spfile;
alter database rename file '+NHCPDB_ARCH/nhcpdb/onlinelog/group_2.258.938775465'
to '+NHCPDB_DATA/nhcpdb/onlinelog/group_2.258.938775465';
alter database rename file '+NHCPDB_ARCH/nhcpdb/onlinelog/group_1.257.938775463'
to '+NHCPDB_DATA/nhcpdb/onlinelog/group_1.257.938775463';
alter database rename file '+NHCPDB_ARCH/nhcpdb/onlinelog/group_3.259.938775599'
to '+NHCPDB_DATA/nhcpdb/onlinelog/group_3.259.938775599';
alter database rename file '+NHCPDB_ARCH/nhcpdb/onlinelog/group_4.260.938775601'
to '+NHCPDB_DATA/nhcpdb/onlinelog/group_4.260.938775601';
4.Restore database
RUN {
allocate channel D1 type 'sbt_tape' parms 'SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.so64';
allocate channel D2 type 'sbt_tape' parms 'SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.so64';
send 'NB_ORA_SERV=nbuserver,NB_ORA_CLIENT=nhcprac1';
restore database; ##无需指定备份片，会自动找最新的备份恢复
RELEASE CHANNEL D1;
RELEASE CHANNEL D2;
}
Recover database
次报错是因为选的时间太短，until time 尽可能选择为最近备份之后的时间
RMAN> RUN {
2> allocate channel D1 type 'sbt_tape' parms 'SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.so64';
3> allocate channel D2 type 'sbt_tape' parms 'SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.so64';
4> send 'NB_ORA_SERV=nbuserver,NB_ORA_CLIENT=nhcprac1';
5> sql 'alter session set nls_date_format= "YYYY-MM-DD HH24:MI:SS"';
6> set until time '2020-02-08 01:05:00';
7> recover database;
8> RELEASE CHANNEL D1;
9> RELEASE CHANNEL D2;
10> }
using target database control file instead of recovery catalog
allocated channel: D1
channel D1: SID=34 instance=nhcpdb1 device type=SBT_TAPE
channel D1: Veritas NetBackup for Oracle - Release 8.1 (2017091017)
allocated channel: D2
channel D2: SID=50 instance=nhcpdb1 device type=SBT_TAPE
channel D2: Veritas NetBackup for Oracle - Release 8.1 (2017091017)
sent command to channel: D1
sent command to channel: D2
sql statement: alter session set nls_date_format= "YYYY-MM-DD HH24:MI:SS"
executing command: SET until clause
Starting recover at 16-FEB-2020 23:35:56
starting media recovery
channel D1: starting archived log restore to default destination
channel D1: restoring archived log
archived log thread=2 sequence=25286
channel D1: restoring archived log
archived log thread=1 sequence=23821
channel D1: restoring archived log
archived log thread=1 sequence=23822
channel D1: restoring archived log
archived log thread=2 sequence=25287
channel D1: reading from backup piece al_6307_1_1031792542
channel D1: piece handle=al_6307_1_1031792542 tag=TAG20200208T010110
channel D1: restored backup piece 1
channel D1: restore complete, elapsed time: 00:00:15
archived log file name=+NHCPDB_DATA/nhcpdb/archivelog/2020_02_16/thread_1_seq_23821.275.1032564965 thread=1 sequence=23821
archived log file name=+NHCPDB_DATA/nhcpdb/archivelog/2020_02_16/thread_2_seq_25286.276.1032564965 thread=2 sequence=25286
archived log file name=+NHCPDB_DATA/nhcpdb/archivelog/2020_02_16/thread_1_seq_23822.277.1032564967 thread=1 sequence=23822
archived log file name=+NHCPDB_DATA/nhcpdb/archivelog/2020_02_16/thread_2_seq_25287.278.1032564967 thread=2 sequence=25287
unable to find archived log
archived log thread=2 sequence=25288
released channel: D1
released channel: D2
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 02/16/2020 23:36:15
RMAN-06054: media recovery requesting unknown archived log for thread 2 with sequence 25288 and starting SCN of 639257332959
###因为选择的时候过新，需要最新的log 只能修复到提示的scn，所以按scn恢复
RMAN> RUN {
2> allocate channel D1 type 'sbt_tape' parms 'SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.so64';
3> allocate channel D2 type 'sbt_tape' parms 'SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.so64';
4> send 'NB_ORA_SERV=nbuserver,NB_ORA_CLIENT=nhcprac1';
5> recover database until scn 639257332959;
6> RELEASE CHANNEL D1;
7> RELEASE CHANNEL D2;
8> }
allocated channel: D1
channel D1: SID=34 instance=nhcpdb1 device type=SBT_TAPE
channel D1: Veritas NetBackup for Oracle - Release 8.1 (2017091017)
allocated channel: D2
channel D2: SID=50 instance=nhcpdb1 device type=SBT_TAPE
channel D2: Veritas NetBackup for Oracle - Release 8.1 (2017091017)
sent command to channel: D1
sent command to channel: D2
Starting recover at 16-FEB-2020 23:41:54
starting media recovery
media recovery complete, elapsed time: 00:00:00
Finished recover at 16-FEB-2020 23:41:55
released channel: D1
released channel: D2
RMAN> alter database open resetlogs;
database opened
RMAN>
- Open resetlog
- 另外一个节点启动时， 也需要先修改这些参数才能启动
SQL> startup
ORACLE instance started.
Total System Global Area 6547296256 bytes
Fixed Size 2265544 bytes
Variable Size 3405778488 bytes
Database Buffers 3120562176 bytes
Redo Buffers 18690048 bytes
ORA-01105: mount is incompatible with mounts by other instances
ORA-19808: recovery destination parameter mismatch
SQL> startup mount;
ORA-01081: cannot start already-running ORACLE - shut it down first
SQL> shutdown immediate;
ORA-01507: database not mounted
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.
Total System Global Area 6547296256 bytes
Fixed Size 2265544 bytes
Variable Size 3456110136 bytes
Database Buffers 3070230528 bytes
Redo Buffers 18690048 bytes
ORA-01105: mount is incompatible with mounts by other instances
ORA-19808: recovery destination parameter mismatch
SQL> show parameter log
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
_use_adaptive_log_file_sync string FALSE
audit_syslog_level string
commit_logging string
db_create_online_log_dest_1 string
db_create_online_log_dest_2 string
db_create_online_log_dest_3 string
db_create_online_log_dest_4 string
db_create_online_log_dest_5 string
enable_ddl_logging boolean FALSE
log_archive_config string
log_archive_dest string
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_1 string LOCATION=+NHCPDB_ARCH
log_archive_dest_10 string
log_archive_dest_11 string
log_archive_dest_12 string
log_archive_dest_13 string
log_archive_dest_14 string
log_archive_dest_15 string
log_archive_dest_16 string
log_archive_dest_17 string
log_archive_dest_18 string
log_archive_dest_19 string
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_2 string
log_archive_dest_20 string
log_archive_dest_21 string
log_archive_dest_22 string
log_archive_dest_23 string
log_archive_dest_24 string
log_archive_dest_25 string
log_archive_dest_26 string
log_archive_dest_27 string
log_archive_dest_28 string
log_archive_dest_29 string
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_3 string
log_archive_dest_30 string
log_archive_dest_31 string
log_archive_dest_4 string
log_archive_dest_5 string
log_archive_dest_6 string
log_archive_dest_7 string
log_archive_dest_8 string
log_archive_dest_9 string
log_archive_dest_state_1 string enable
log_archive_dest_state_10 string enable
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_state_11 string enable
log_archive_dest_state_12 string enable
log_archive_dest_state_13 string enable
log_archive_dest_state_14 string enable
log_archive_dest_state_15 string enable
log_archive_dest_state_16 string enable
log_archive_dest_state_17 string enable
log_archive_dest_state_18 string enable
log_archive_dest_state_19 string enable
log_archive_dest_state_2 string enable
log_archive_dest_state_20 string enable
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_state_21 string enable
log_archive_dest_state_22 string enable
log_archive_dest_state_23 string enable
log_archive_dest_state_24 string enable
log_archive_dest_state_25 string enable
log_archive_dest_state_26 string enable
log_archive_dest_state_27 string enable
log_archive_dest_state_28 string enable
log_archive_dest_state_29 string enable
log_archive_dest_state_3 string enable
log_archive_dest_state_30 string enable
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_state_31 string enable
log_archive_dest_state_4 string enable
log_archive_dest_state_5 string enable
log_archive_dest_state_6 string enable
log_archive_dest_state_7 string enable
log_archive_dest_state_8 string enable
log_archive_dest_state_9 string enable
log_archive_duplex_dest string
log_archive_format string %t_%s_%r.dbf
log_archive_local_first boolean TRUE
log_archive_max_processes integer 4
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
log_archive_min_succeed_dest integer 1
log_archive_start boolean FALSE
log_archive_trace integer 0
log_buffer integer 18022400
log_checkpoint_interval integer 0
log_checkpoint_timeout integer 1800
log_checkpoints_to_alert boolean FALSE
log_file_name_convert string
remote_login_passwordfile string EXCLUSIVE
sec_case_sensitive_logon boolean FALSE
sec_max_failed_login_attempts integer 10
SQL>
SQL> alter system set log_archive_dest_1="Location=+NHCPDB_DATA" scope=spfile; --修改归档路径 删掉故障DG
System altered.
SQL>
SQL> shutdown immediate;
ORA-01507: database not mounted
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.
Total System Global Area 6547296256 bytes
Fixed Size 2265544 bytes
Variable Size 3456110136 bytes
Database Buffers 3070230528 bytes
Redo Buffers 18690048 bytes
ORA-01105: mount is incompatible with mounts by other instances
ORA-19808: recovery destination parameter mismatch
SQL>
SQL> show parameter control_file
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
control_file_record_keep_time integer 7
control_files string +NHCPDB_DATA/nhcpdb/controlfil
e/current.260.938775459
SQL>
SQL> show recover
SP2-0158: unknown SHOW option "recover"
SQL> show parameter recover
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest string +NHCPDB_ARCH
db_recovery_file_dest_size big integer 4407M
db_unrecoverable_scn_tracking boolean TRUE
recovery_parallelism integer 0
SQL>
SQL> alter system set db_recovery_file_dest='' scope=spfile;
System altered.
SQL> shutdown immediate;
ORA-01507: database not mounted
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.
Total System Global Area 6547296256 bytes
Fixed Size 2265544 bytes
Variable Size 3456110136 bytes
Database Buffers 3070230528 bytes
Redo Buffers 18690048 bytes
Database mounted.
SQL> alter database open;
Database altered.
SQL>
SQL>
SQL>
附录：NBU 异机恢复需要满足一下条件
同平台同数据库版本
