Part1问题
业务反馈说连不上数据库,Ora报错
2025-07-31T09:55:50.160256024+08:00 Caused by: java.sql.SQLRecoverableException: ORA-01033: ORACLE initialization or shutdown in progress
Part2定位
查看数据库报警日志所在位置
SQL> set pages 999 lines 999
SQL> select NAME,VALUE from v$diag_info where NAME='Diag Trace';
NAME VALUE
--------------- ---------------------------------------------------
Diag Trace /u01/app/oracle/diag/rdbms/db1/db11/trace
重启前的操作,一看就是adg库的设置操作,注意log_archive_dest_1的操作,这是我后续刷一遍才找到的
2025-07-31T08:35:36.713535+08:00
ALTER DATABASE FORCE LOGGING
2025-07-31T08:35:36.727929+08:00
ALTER DATABASE FORCE LOGGING command is waiting for existing direct writes to finish. This may take a long time.
Completed: ALTER DATABASE FORCE LOGGING
.... (PID:8136): Enable RFS client [krsr.c:18167]
2025-07-31T08:35:36.854416+08:00
ALTER SYSTEM SET log_archive_config='DG_CONFIG=(db1,db1dg)' SCOPE=BOTH SID='*';
.... (PID:8136): Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST [krsd.c:18436]
***2025-07-31T08:35:36.883530+08:00
ALTER SYSTEM SET log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=db1' SCOPE=BOTH SID='*';
.... (PID:8136): Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST [krsd.c:18436]***
2025-07-31T08:35:36.916296+08:00
ALTER SYSTEM SET log_archive_dest_2='SERVICE=db1dg ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=db1dg' SCOPE=BOTH SID='*';
2025-07-31T08:35:36.945858+08:00
ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=BOTH SID='*';
2025-07-31T08:35:36.973474+08:00
ALTER SYSTEM SET fal_server='DB1DG' SCOPE=BOTH SID='*';
2025-07-31T08:35:37.000466+08:00
ALTER SYSTEM SET fal_client='DB1' SCOPE=BOTH SID='*';
2025-07-31T08:35:37.019056+08:00
log_archive_dest_1 设置主备库的对应关系,但是和平常不一样的是,他的归档路径设置的是闪回的路径,而不是直接设置的归档路径
数据库一直在重复报警,其中最关键的几个是 ORA-19801: initialization parameter DB_RECOVERY_FILE_DEST is not set
ORA-16032: parameter LOG_ARCHIVE_DEST_1 destination string cannot be translated
但是没看到数据库宕机的信息,当时以为是跟数据库宕机无关
2025-07-31T08:35:37.417322+08:00
TT04 (PID:8153): Error 12514 received logging on to the standby
2025-07-31T08:35:37.418003+08:00
Errors in file /u01/app/oracle/diag/rdbms/db1/db11/trace/db11_tt04_8153.trc:
ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
TT04 (PID:8153): Error 12514 for LNO:2 to 'db1dg'
2025-07-31T08:35:38.207741+08:00
Thread 1 advanced to log sequence 2581 (LGWR switch), current SCN: 18503093708396
Current log# 1 seq# 2581 mem# 0: +REDO/DB1/ONLINELOG/group_1.257.1185026041
2025-07-31T08:35:38.265099+08:00
Errors in file /u01/app/oracle/diag/rdbms/db1/db11/trace/db11_arc0_19954.trc:
ORA-19801: initialization parameter DB_RECOVERY_FILE_DEST is not set
2025-07-31T08:35:38.269714+08:00
Errors in file /u01/app/oracle/diag/rdbms/db1/db11/trace/db11_arc0_19954.trc:
ORA-16032: parameter LOG_ARCHIVE_DEST_1 destination string cannot be translated
ORA-19801: initialization parameter DB_RECOVERY_FILE_DEST is not set
2025-07-31T08:35:38.270001+08:00
ARC0 (PID:19954): Stuck archiver: insufficient local LADs
ARC0 (PID:19954): Stuck archiver condition declared
2025-07-31T08:35:38.329080+08:00
Errors in file /u01/app/oracle/diag/rdbms/db1/db11/trace/db11_arc0_19954.trc:
ORA-16038: log 2 sequence# 2580 cannot be archived
ORA-00312: online log 2 thread 1: '+REDO/DB1/ONLINELOG/group_2.258.1185026041'
ORA-16032: parameter LOG_ARCHIVE_DEST_1 destination string cannot be translated
ORA-19801: initialization parameter DB_RECOVERY_FILE_DEST is not set
ARC0 (PID:19954): Archival error occurred on a closed thread, archiver continuing
2025-07-31T08:35:38.329569+08:00
ORACLE Instance db11, archival error, archiver continuing
2025-07-31T08:35:38.712395+08:00
ALTER SYSTEM SET remote_listener='db1-scan:1521' SCOPE=MEMORY SID='db11';
2025-07-31T08:35:38.716984+08:00
ALTER SYSTEM SET listener_networks='' SCOPE=MEMORY SID='db11';
2025-07-31T08:40:37.517207+08:00
2025-07-31T08:50:38.490111+08:00
TT00 (PID:19952): Error 12514 received logging on to the standby
2025-07-31T08:51:38.504528+08:00
Errors in file /u01/app/oracle/diag/rdbms/db1/db11/trace/db11_tt00_19952.trc:
ORA-19801: initialization parameter DB_RECOVERY_FILE_DEST is not set
2025-07-31T08:55:38.617757+08:00
终于看到数据库宕机的关键信息,他和闪回参数是在一起的,所以下意识认为此次宕机和闪回有关,又有人动了闪回,上边的adg配置报错只是障眼法
2025-07-31T09:40:42.306102+08:00
TT00 (PID:19952): Error 12514 received logging on to the standby
2025-07-31T09:41:42.324409+08:00
Errors in file /u01/app/oracle/diag/rdbms/db1/db11/trace/db11_tt00_19952.trc:
ORA-19801: initialization parameter DB_RECOVERY_FILE_DEST is not set
2025-07-31T09:45:47.293322+08:00
TT00 (PID:19952): Error 1017 received logging on to the standby
TT00 (PID:19952): -------------------------------------------------------------------------
TT00 (PID:19952): Check that the source and target databases are using a password file
TT00 (PID:19952): and remote_login_passwordfile is set to SHARED or EXCLUSIVE,
TT00 (PID:19952): and that the SYS password is same in the password files,
TT00 (PID:19952): returning error ORA-16191
TT00 (PID:19952): -------------------------------------------------------------------------
2025-07-31T09:46:47.309819+08:00
Errors in file /u01/app/oracle/diag/rdbms/db1/db11/trace/db11_tt00_19952.trc:
ORA-19801: initialization parameter DB_RECOVERY_FILE_DEST is not set
2025-07-31T09:49:34.897566+08:00
Shutting down ORACLE instance (immediate) (OS id: 20597)
2025-07-31T09:49:34.903531+08:00
Shutdown is initiated by oraagent.bin@db11 (TNS V1-V3).
Stopping background process SMCO
2025-07-31T09:49:35.983147+08:00
Shutting down instance: further logons disabled
2025-07-31T09:49:36.197634+08:00
Stopping background process CJQ0
Stopping background process MMNL
Stopping background process MMON
2025-07-31T09:49:38.749804+08:00
License high water mark = 269
2025-07-31T09:49:38.821545+08:00
Dispatchers and shared servers shutdown
ORA-19801: initialization parameter DB_RECOVERY_FILE_DEST is not set
可以看到就是因为他导致的,但是真实原因是这样吗?
重启4~5次
2025-07-31T09:56:19.106051+08:00
ARC1 (PID:64761): Error 1017 received logging on to the standby
ARC1 (PID:64761): -------------------------------------------------------------------------
ARC1 (PID:64761): Check that the source and target databases are using a password file
ARC1 (PID:64761): and remote_login_passwordfile is set to SHARED or EXCLUSIVE,
ARC1 (PID:64761): and that the SYS password is same in the password files,
ARC1 (PID:64761): returning error ORA-16191
ARC1 (PID:64761): -------------------------------------------------------------------------
ARC1 (PID:64761): FAL: Error 16191 connecting to DB1DG for fetching gap sequence
2025-07-31T09:56:19.118051+08:00
Errors in file /u01/app/oracle/diag/rdbms/db1/db11/trace/db11_arc1_64761.trc:
ORA-16191: Primary log shipping client not logged on standby
恢复正常以后还是再刷密码文件错误的问题
Part3解决(事后判断,因为已经被人解决了)
先判断闪回是否开启,没开
SQL> select flashback_on from v$database;
FLASHBACK_ON
------------------
NO
再看看闪回的参数,没设置路径
SQL> show parameter DB_RECOVERY_FILE_DEST
NAME TYPE VALUE
--------------------------- ----------- -------------
db_recovery_file_dest string
db_recovery_file_dest_size big integer 0
确定数据库参数的具体数值
SQL> create pfile ='/tmp/ora_20250731.ora' from spfile;
##这俩应该是有用的
*.log_archive_dest_1='LOCATION=+arch VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=db1'
*.log_archive_dest_2='SERVICE=db1dg ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=db1dg'
后续是好了,但是不是我操作的,我觉得是关闭了闪回(?)
alter database flashback OFF;
最后觉得是 log_archive_dest_1 有问题,接着网上翻日志
2025-07-31T08:35:36.883530+08:00
ALTER SYSTEM SET log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=db1' SCOPE=BOTH SID='*';
.... (PID:8136): Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST [krsd.c:18436]
Part4回顾(必看)
1、业务反馈异常,登录数据库判断数据库信息(状态、启动时间)
2、定位alert的路径,抓取宕机前的报警,判断是搭建adg导致的异常重启
3、根据alert报警闪回的路径查看闪回的参数,发现没开启闪回,也没设置相关的参数,以为是后来关闭的
4、定位闪回的操作(alter system set 或者 log_archive_dest_1),没发现相关的命令
5、查看 闪回的概念,在测试环境做实验,发现闪回除了爆满,不会导致数据库异常重启,没有发现这个报错,也没发现相关操作
6、意识到不是根本原因,接着查看宕机前的操作命令,查看关键字 USE_DB_RECOVERY_FILE_DEST ,发现了设置 log_archive_dest_1 的命令,log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST
7、最后梳理 ,相关操作人员搭建ADG备库,在设置 log_archive_dest_1 的时候没有指定具体的路径,而是制定了闪回的参数,但是闪回没有开启,没有设置闪回存放的路径,所以ADG搭建失败,数据库报错归档异常,报错USE_DB_RECOVERY_FILE_DEST参数未设置,导致数据库异常重启4-5次,最后重新设置了log_archive_dest_1正确的归档路径才恢复正常。
Part5原理
5.1闪回的注意事项
闪回的开启必须基于存档的开启 但是开了归档不相当于开了闪回
5.2adg的注意事项
log_archive_dest_1参数里的归档路径,可以设置具体路径,也可以设置参数,但是设置具体路径更安全
5.3闲聊
谁说没枪头就捅不死人
谁说adg搭建对主库没影响




