暂无图片
暂无图片
1
暂无图片
暂无图片
暂无图片
首页 / Every Day of a DBA，第144期: Windows 19c GI ASM 密码文件错误

Every Day of a DBA，第144期: Windows 19c GI ASM 密码文件错误

ByteHouse 19小时前
28

作者：bytehouse Oracle ACE、PostgreSQL ACE
10+年数据库架构与运维实战经验
公众号：bytehouse
墨天轮专栏：bytehouse
CSDN：Young DBA

摘要：

Every Day of a DBA，第142期: Oracle 19c RAC ASM密码文件丢失/损坏恢复

1. 故障概述

1.1 环境描述

  • 操作系统：Windows Server 2019 双节点

  • 数据库版本：Oracle 19.3.0 Grid Infrastructure + RAC

  • 存储架构：OCR/Voting磁盘组部署在ASM磁盘组+OCR，ASM密码文件默认存放于共享ASM磁盘组

  • 故障现象：业务更新程序后重启两台主机，节点zxrdsrv2集群服务无法启动，CRSD进程直接PANIC退出，集群整体异常，数据库无法对外提供服务；另一节点集群服务运行正常

  • 核心报错日志：CRSD.trc日志持续抛出ORA-01017、PROC-26错误，最终CRSD进程异常退出

1.2 故障影响范围

  • 故障节点：所有集群资源、ASM实例、数据库实例全部无法启动

  • 集群层面：节点心跳异常，集群拓扑异常，无法进行资源切换

  • 业务层面：单节点数据库服务中断，无法自动故障转移

2. 故障日志

Trace file E:\app\grid\diag\crs\zxrdsrv2\crs\trace\crsd.trc
Oracle Database 19c Clusterware Release 19.0.0.0.0 - Production
Version 19.3.0.0.0 Copyright 1996, 2019 Oracle. All rights reserved.
 default:16792: 1: clskec:has:CLSU:910 4 args[clsdAdr_CLSK_err][mod=clsdadr.c][loc=(:CLSD00302:)][msg=2026-06-18 01:00:13.411 (:CLSD00302:) Trace file size and number of segments fetched from environment variable: ORA_DAEMON_TRACE_FILE_OPTIONS filesize=26214400,numsegments=10    Detected in function clsdAdrGetEnvVar_TFO at line number 6819. ]

2026-06-18 01:00:14.561 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 0000017062533B00 [0000000000000164] { gipcReceiveRequest : peerName 'tcp', data 00000170621EB588, len 48, olen 48, off 0, parentEndp 00000170625330F0, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 00000170625330F0 [0000000000000146] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57628)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61012)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 00000170624CEFE0, sendp 0000017062534510 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x10000 }
2026-06-18 01:00:14.576 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 0000017062533B00 [00000000000001ab] { gipcReceiveRequest : peerName 'tcp', data 00000170621EB318, len 48, olen 48, off 0, parentEndp 00000170625330F0, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 00000170625330F0 [000000000000018d] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57629)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61012)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 00000170624CFAD0, sendp 0000017062534510 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x10000 }
2026-06-18 01:00:14.584 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 0000017062307C20 [00000000000001dc] { gipcReceiveRequest : peerName 'tcp', data 00000170621EC698, len 48, olen 48, off 0, parentEndp 0000017062306EB0, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062306EB0 [00000000000001be] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57630)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61012)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 00000170624CFD00, sendp 00000170623082D0 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x10000 }
2026-06-18 01:00:14.876 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170623103F0 [0000000000000232] { gipcReceiveRequest : peerName 'tcp', data 00000170621EB318, len 48, olen 48, off 0, parentEndp 0000017062534070, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062534070 [0000000000000214] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57635)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61012)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 000001706231FBC0, sendp 000001706231D860 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x10000 }
    CLSB:16792: [     INFO] Argument count (argc) for this daemon is 2
    CLSB:16792: [     INFO] Argument 0 is: E:\app\19.3.0\grid\bin\crsd.exe
    CLSB:16792: [     INFO] Argument 1 is: reboot
2026-06-18 01:00:14.893 : GIPCPKT:15888:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170623119E0 [000000000000027c] { gipcReceiveRequest : peerName 'tcp', data 00000170621EC908, len 48, olen 48, off 0, parentEndp 0000017062624360, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062624360 [000000000000025e] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57636)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61011)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 16308, readyRef 0000000000000000, ready 0, wobj 000001706231F060, sendp 0000017062628FC0 status 13flags 0x2008a712, flags-2 0x100, usrFlags 0x30020 }
2026-06-18 01:00:14.893 : GIPCPKT:15888:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170623119E0 [0000000000000286] { gipcReceiveRequest : peerName 'tcp', data 0000017062616488, len 153, olen 153, off 0, parentEndp 0000017062624360, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062624360 [000000000000025e] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57636)(GIPCID=8c227d89-faaa2060-15136))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61011)(GIPCID=faaa2060-8c227d89-16308))', numPend 6, numReady 2, numDone 1, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 16308, readyRef 0000000000000000, ready 0, wobj 000001706231F060, sendp 0000017062628FC0 status 13flags 0x2008a712, flags-2 0x100, usrFlags 0x30020 }
2026-06-18 01:00:14.908 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 000001706230D3C0 [00000000000002dc] { gipcReceiveRequest : peerName 'tcp', data 00000170621EB658, len 48, olen 48, off 0, parentEndp 0000017062943550, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062943550 [00000000000002c1] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57637)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61100)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 18924, readyRef 0000000000000000, ready 0, wobj 000001706231F1B0, sendp 0000017062638540 status 13flags 0x202ca712, flags-2 0x100, usrFlags 0x34000 }
2026-06-18 01:00:14.913 : CRSMAIN:16792: [     INFO]  First attempt: init CSS context succeeded.
2026-06-18 01:00:14.913 : CRSMAIN:16792: [     INFO]  Start mode: normal
2026-06-18 01:00:14.918 :  CLSDMT:17996: [     INFO] PID for the Process [15136], connkey CRSD
2026-06-18 01:00:15.312 : GIPCPKT:17996:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170629FEA80 [0000000000000341] { gipcReceiveRequest : peerName 'ipc', data 000001706294FBE8, len 68, olen 68, off 0, parentEndp 0000017062950DB0, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062950DB0 [000000000000031f] { gipcEndpoint : localAddr 'ipc://zxrdsrv2_CRSD', remoteAddr 'ipc://zxrdsrv2_CRSD', numPend 4, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 11428, readyRef 0000000000000000, ready 0, wobj 000001706231F760, sendp 0000017062981A40 status 0flags 0x2010a616, flags-2 0x80, usrFlags 0x20020 }
2026-06-18 01:00:15.312 : CRSMAIN:16792: [     INFO]  CRS Daemon Starting
2026-06-18 01:00:15.312 : CRSMAIN:17996: [     INFO]  Process environment is not initialized yet!
2026-06-18 01:00:15.316 : CRSMAIN:17996: [     INFO]  Process environment is not initialized yet!
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: clsdadr  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: clsdstat  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: clsdnreg  0
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: clsdynam  0
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: allcomp  0
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: default  0
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCLIB  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCXBAD  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCLXPT  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCUNDE  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPC  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCGEN  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCTRAC  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCWAIT  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCXCPT  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCOSD  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCBASE  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCCLSA  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCCLSC  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCEXMP  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCGMOD  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCHEAD  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCMUX  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCNET  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCNULL  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCPKT  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCSMEM  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCHAUP  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCHALO  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCHTHR  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCHGEN  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCHLCK  2
2026-06-18 01:00:15.318 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCHDEM  2
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCHWRK  2
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCTLS  2
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: GIPCHGNS  2
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: GPNP  0
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: CLSINET  0
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: RD  0
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: OCRAPI  1
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: OCRUTL  1
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: OCRMSG  1
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: OCRCLI  1
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: CLSXSIG  0
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: CLWAL  1
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: CLSXSEC  0
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: clsdimt  0
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: CLSCAL  0
2026-06-18 01:00:15.319 :    CRSD:16792: [     INFO]  Logging level for Module: CLSB  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CLSCEVT  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: COMMCRS  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: COMMNS  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CSSCLNT  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSMAIN  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: clsdmt  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: clsdms  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSUI  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSCOMM  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSRTI  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSPLACE  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSAPP  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSRES  5
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSTIMER  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSEVT  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSD  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CLUCLS  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CLSVER  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CLSFRAME  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSPE  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSSE  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSRPT  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSOCR  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: UiServer  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: AGFW  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: SuiteTes  1
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSSHARE  1
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSSEC  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSCCL  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: CRSCEVT  0
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: AGENT  1
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: OCRSRV  1
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: OCRMAS  1
2026-06-18 01:00:15.320 :    CRSD:16792: [     INFO]  Logging level for Module: OCRCAC  1
2026-06-18 01:00:15.321 :    CRSD:16792: [     INFO]  Logging level for Module: OCRRAW  1
2026-06-18 01:00:15.321 :    CRSD:16792: [     INFO]  Logging level for Module: OCROSD  1
2026-06-18 01:00:15.321 :    CRSD:16792: [     INFO]  Logging level for Module: OCRASM  1
2026-06-18 01:00:15.321 :    CRSD:16792: [     INFO]  Logging level for Module: CLSO  0
2026-06-18 01:00:15.321 :    CRSD:16792: [     INFO]  Logging level for Module: SCLSOPCT  0
2026-06-18 01:00:15.321 : CRSMAIN:16792: [     INFO]  Checking the OCR device
2026-06-18 01:00:15.322 : CRSMAIN:16792: [     INFO]  Connecting to the CSS Daemon
2026-06-18 01:00:15.322 : CRSMAIN:16792: [     INFO]  Getting local node number
2026-06-18 01:00:15.322 : GIPCPKT:17996:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170629FD130 [000000000000036b] { gipcReceiveRequest : peerName 'ipc', data 0000017061393138, len 27, olen 27, off 0, parentEndp 000001706294FFD0, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 000001706294FFD0 [0000000000000337] { gipcEndpoint : localAddr 'ipc://zxrdsrv2_CRSD', remoteAddr 'ipc://zxrdsrv2_CRSD', numPend 4, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 11428, readyRef 0000000000000000, ready 0, wobj 000001706231FDF0, sendp 00000170629FCC40 status 0flags 0x2010a616, flags-2 0x80, usrFlags 0x20020 }
2026-06-18 01:00:15.324 : CRSMAIN:16792: [     INFO]  Initializing OCR
2026-06-18 01:00:15.324 : default:16792: proa_init: OCR Abstraction layer initialization. Bootlevel:[1]
2026-06-18 01:00:15.350 :  OCRAPI:16792: a_init: Successfully initialized the patch management context.
2026-06-18 01:00:15.357 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170629FD130 [000000000000039b] { gipcReceiveRequest : peerName 'tcp', data 000001706260D6B8, len 48, olen 48, off 0, parentEndp 0000017062951000, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062951000 [000000000000037d] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57638)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61012)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 00000170613CCC80, sendp 0000017062DBE760 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x10000 }
2026-06-18 01:00:15.367 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 0000017062310FC0 [00000000000003d6] { gipcReceiveRequest : peerName 'tcp', data 000001706260CC28, len 48, olen 48, off 0, parentEndp 00000170629514A0, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 00000170629514A0 [00000000000003b8] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57639)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61013)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 00000170613CCCF0, sendp 000001706247EDC0 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x0 }
2026-06-18 01:00:15.390 :  OCRAPI:16792: a_init: Node role retrieved is [1] and the type is [3].
2026-06-18 01:00:15.391 :  OCRAPI:16792: a_init: ASM mode retrieved is [2]
2026-06-18 01:00:15.393 :  OCRAPI:16792: a_init:13: Clusterware init successful
2026-06-18 01:00:15.393 :  OCRAPI:16792: a_init:26: This node is functioning in role [1].
2026-06-18 01:00:15.393 :  OCRAPI:16792: a_init:15: Successfully initialized the Cache layer.
2026-06-18 01:00:15.400 :  OCRRAW:16792: proprioo: opening OCR device(s)
2026-06-18 01:00:15.401 :  OCRRAW:16792: proprioo: Successfully opened the non-ASM locations if configured.
2026-06-18 01:00:15.403 :  OCRRAW:16792: kgfnInitEnv env=00000034672FB3C8 flags=0x0

2026-06-18 01:00:15.403 :  OCRRAW:16792: kgfoCreateCtxExt2 trcflg: 0 [trclvl_in:3] ctx:0000017062E2D190

2026-06-18 01:00:15.405 :  OCRRAW:16792: kgfnInitEnv env=00000034672FB3C8 flags=0x0

2026-06-18 01:00:15.406 :  OCRRAW:16792: kgfoCreateCtxExt2 trcflg: 0 [trclvl_in:3] ctx:0000017062ECAD00

2026-06-18 01:00:15.417 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170629FD130 [0000000000000433] { gipcReceiveRequest : peerName 'tcp', data 000001706260D928, len 48, olen 48, off 0, parentEndp 00000170629516F0, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 00000170629516F0 [0000000000000421] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57640)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61013)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 0000017062F18430, sendp 0000017062535020 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x0 }
2026-06-18 01:00:15.442 :  OCRRAW:16792: kgfnFindLocalNode03: kgfn_find_node_sid found no members

2026-06-18 01:00:15.442*:kgfn.c@1412: kgfnFindLocalNode: found no members
2026-06-18 01:00:15.442 :  OCRRAW:16792: kgfnFindLocalNode: not ok

2026-06-18 01:00:15.442*:kgfn.c@1466: kgfnFindLocalNode: not ok
2026-06-18 01:00:15.442 :  OCRRAW:16792: kgfnTgtInit: local node not found, free kgfnpds

2026-06-18 01:00:15.442*:kgfn.c@2252: kgfnTgtInit: not found
2026-06-18 01:00:15.442 :  OCRRAW:16792: kgfnGetBeqData failed init target; inst=(null) flags=0x6000

2026-06-18 01:00:15.442*:kgfn.c@5941: kgfnGetBeqData: kgfnTgtInit failed, inst=NULL flags=0x6000
2026-06-18 01:00:15.449 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170629FD130 [0000000000000472] { gipcReceiveRequest : peerName 'tcp', data 000001706260DD38, len 48, olen 48, off 0, parentEndp 00000170629516F0, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 00000170629516F0 [0000000000000454] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57641)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61012)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 0000017062F20F00, sendp 00000170625362C0 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x10000 }
2026-06-18 01:00:15.456 :   CLSNS:16792: clsns_SetTraceLevel:trace level set to 1.
2026-06-18 01:00:15.686 : default:16792: Inited LSF context: 0000017062F15C40 
2026-06-18 01:00:15.910 : CLSCRED:16792: clsCredCommonInit: Inited singleton credctx.
2026-06-18 01:00:15.910 : CLSCRED:16792: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2026-06-18 01:00:15.973 :  OCRRAW:16792: 9348 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2026-06-18 01:00:15.984 :  OCRRAW:16792: 9348 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2026-06-18 01:00:16.087 : CLSCRED:16792: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.b6f94c9a7d4f5fe9ff504135765d81e7.system not found
2026-06-18 01:00:16.088 :  OCRRAW:16792: 9066 Error 4 opening dom system in 000001706138EFD0

2026-06-18 01:00:16.106 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170630CD680 [0000000000000a63] { gipcReceiveRequest : peerName 'tcp', data 000001706260DC68, len 48, olen 48, off 0, parentEndp 0000017062951B90, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062951B90 [0000000000000a51] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57648)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61013)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 0000017062F20D40, sendp 0000017062EF08C0 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x0 }
2026-06-18 01:00:16.132 :  OCRRAW:16792: kgfnConnect2: kgfnGetBeqData failed

2026-06-18 01:00:16.132*:kgfn.c@5129: kgfnConnect2: kgfnGetBeqData failed
2026-06-18 01:00:16.238 :  OCRRAW:16792: kgfnConnect2Int: cstr=(DESCRIPTION=(TCP_USER_TIMEOUT=1)(TRANSPORT_CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=193.169.100.10)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))

2026-06-18 01:00:16.238*:kgfn.c@6929: kgfnConnect2Int: cstr=(DESCRIPTION=(TCP_USER_TIMEOUT=1)(TRANSPORT_CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=193.169.100.10)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))
2026-06-18 01:00:16.238*:kgfn.c@7107: kgfnConnect2Int: OCISessionBegin failed
2026-06-18 01:00:17.638 :  OCRRAW:16792: kgfnRecordErr 1017 OCI error:
ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:17.638*:kgfn.c@1784: kgFailed to connect to ASM 1 1 0 0000000000000000 0
Failed to connect to ASM 1 1 0 0000000000000000 0
Failed to connect to ASM 1 1 0 0000000000000000 0
ed 00000170625365B0
2026-06-18 01:00:17.645 :  OCRRAW:16792: kgfnConnect2: failed to connect

2026-06-18 01:00:17.645*:kgfn.c@5461: kgfnConnect2: failed to connect
2026-06-18 01:00:17.645 :  OCRRAW:16792: kgfnConnect2Retry: failed to connect connect after 1 attempts, 224s elapsed

2026-06-18 01:00:17.645 :  OCRRAW:16792: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:17.645*:kgfo.c@1003: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied

2026-06-18 01:00:17.645 :  OCRRAW:16792: -- trace dump on error exit --

2026-06-18 01:00:17.645 :  OCRRAW:16792: Error [kgfoAl06] in [kgfokge] at kgfo.c:2165

2026-06-18 01:00:17.645 :  OCRRAW:16792: ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:17.645 :  OCRRAW:16792: Category: 7

2026-06-18 01:00:17.645 :  OCRRAW:16792: DepInfo: 1017

2026-06-18 01:00:17.646 :  OCRRAW:16792: -- trace dump end --

2026-06-18 01:00:17.646 :  OCRASM:16792: proprasmo: Failed to open the file in DG [OCR]
2026-06-18 01:00:17.646 :  OCRASM:16792: proprasmo: Error in open/create file in dg [OCR]
  OCRASM:16792: SLOS : SLOS: cat=7, opn=kgfoAl06, dep=1017, loc=kgfokge

2026-06-18 01:00:17.646 :  OCRASM:16792: ASM Error Stack : ORA-01017: invalid username/password; logon denied

 default:16792: u_set_gbl_comp_error: comptype '108' : error '7'
2026-06-18 01:00:17.658 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170624B8630 [0000000000000c19] { gipcReceiveRequest : peerName 'tcp', data 000001706260E218, len 48, olen 48, off 0, parentEndp 0000017062951B90, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062951B90 [0000000000000c07] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57668)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61013)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 0000017062F21520, sendp 0000017062DC7EC0 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x0 }
2026-06-18 01:00:17.684 :  OCRRAW:16792: kgfnFindLocalNode03: kgfn_find_node_sid found no members

2026-06-18 01:00:17.684*:kgfn.c@1412: kgfnFindLocalNode: found no members
2026-06-18 01:00:17.684 :  OCRRAW:16792: kgfnFindLocalNode: not ok

2026-06-18 01:00:17.684*:kgfn.c@1466: kgfnFindLocalNode: not ok
2026-06-18 01:00:17.684 :  OCRRAW:16792: kgfnTgtInit: local node not found, free kgfnpds

2026-06-18 01:00:17.684*:kgfn.c@2252: kgfnTgtInit: not found
2026-06-18 01:00:17.684 :  OCRRAW:16792: kgfnGetBeqData failed init target; inst=(null) flags=0x6000

2026-06-18 01:00:17.684*:kgfn.c@5941: kgfnGetBeqData: kgfnTgtInit failed, inst=NULL flags=0x6000
2026-06-18 01:00:17.686 :   CLSNS:16792: clsns_SetTraceLevel:trace level set to 1.
2026-06-18 01:00:17.944 : default:16792: Inited LSF context: 0000017062DB54F0 
2026-06-18 01:00:18.184 : CLSCRED:16792: clsCredCommonInit: Inited singleton credctx.
2026-06-18 01:00:18.184 : CLSCRED:16792: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2026-06-18 01:00:18.254 :  OCRRAW:16792: 9348 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2026-06-18 01:00:18.267 :  OCRRAW:16792: 9348 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2026-06-18 01:00:18.382 : CLSCRED:16792: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.b6f94c9a7d4f5fe9ff504135765d81e7.system not found
2026-06-18 01:00:18.384 :  OCRRAW:16792: 9066 Error 4 opening dom system in 000001706138F2D0

2026-06-18 01:00:18.402 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 000001706302ECB0 [0000000000001204] { gipcReceiveRequest : peerName 'tcp', data 000001706260DC68, len 48, olen 48, off 0, parentEndp 0000017062950910, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062950910 [00000000000011f2] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57671)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61013)(GIPCID=00000000-00000000-0))', numPend 6, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 0000017062F20DB0, sendp 00000170635001C0 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x0 }
2026-06-18 01:00:18.424 :  OCRRAW:16792: kgfnConnect2: kgfnGetBeqData failed

2026-06-18 01:00:18.424*:kgfn.c@5129: kgfnConnect2: kgfnGetBeqData failed
2026-06-18 01:00:18.532 :  OCRRAW:16792: kgfnConnect2Int: cstr=(DESCRIPTION=(TCP_USER_TIMEOUT=1)(TRANSPORT_CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=193.169.100.10)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))

2026-06-18 01:00:18.532*:kgfn.c@6929: kgfnConnect2Int: cstr=(DESCRIPTION=(TCP_USER_TIMEOUT=1)(TRANSPORT_CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=193.169.100.10)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))
2026-06-18 01:00:18.532*:kgfn.c@7107: kgfnConnect2Int: OCISessionBegin failed
2026-06-18 01:00:19.917 :  OCRRAW:16792: kgfnRecordErr 1017 OCI error:
ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:19.917*:kgfn.c@1784: kgfnRecordErrPriv: 1017 error=ORA-01017: invalid username/password; logon denied

2026-06-18 01:00:19.920 : default:16792: clsCredDomClose: Credctx deleted 0000017062535860
2026-06-18 01:00:19.920 :  OCRRAW:16792: kgfnConnect2: failed to connect

2026-06-18 01:00:19.920*:kgfn.c@5461: kgfnConnect2: failed to connect
2026-06-18 01:00:19.920 :  OCRRAW:16792: kgfnConnect2Retry: failed to connect connect after 1 attempts, 227s elapsed

2026-06-18 01:00:19.920 :  OCRRAW:16792: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:19.920*:kgfo.c@1003: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied

2026-06-18 01:00:19.931 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 0000017063030CC0 [00000000000013ba] { gipcReceiveRequest : peerName 'tcp', data 000001706260CDC8, len 48, olen 48, off 0, parentEndp 0000017062950910, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062950910 [00000000000013a8] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57688)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61013)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 0000017062F21280, sendp 0000017062FB0770 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x0 }
2026-06-18 01:00:19.956 :  OCRRAW:16792: kgfnFindLocalNode03: kgfn_find_node_sid found no members

2026-06-18 01:00:19.956*:kgfn.c@1412: kgfnFindLocalNode: found no members
2026-06-18 01:00:19.956 :  OCRRAW:16792: kgfnFindLocalNode: not ok

2026-06-18 01:00:19.956*:kgfn.c@1466: kgfnFindLocalNode: not ok
2026-06-18 01:00:19.956 :  OCRRAW:16792: kgfnTgtInit: local node not found, free kgfnpds

2026-06-18 01:00:19.956*:kgfn.c@2252: kgfnTgtInit: not found
2026-06-18 01:00:19.956 :  OCRRAW:16792: kgfnGetBeqData failed init target; inst=(null) flags=0x6000

2026-06-18 01:00:19.956*:kgfn.c@5941: kgfnGetBeqData: kgfnTgtInit failed, inst=NULL flags=0x6000
2026-06-18 01:00:19.957 :   CLSNS:16792: clsns_SetTraceLevel:trace level set to 1.
2026-06-18 01:00:20.078 : CRSMAIN:17996: [     INFO]  Process environment is not initialized yet!
2026-06-18 01:00:20.122 : default:16792: Inited LSF context: 0000017062DB5410 
2026-06-18 01:00:20.315 : CLSCRED:16792: clsCredCommonInit: Inited singleton credctx.
2026-06-18 01:00:20.315 : CLSCRED:16792: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2026-06-18 01:00:20.386 :  OCRRAW:16792: 9348 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2026-06-18 01:00:20.399 :  OCRRAW:16792: 9348 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2026-06-18 01:00:20.514 : CLSCRED:16792: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.b6f94c9a7d4f5fe9ff504135765d81e7.system not found
2026-06-18 01:00:20.515 :  OCRRAW:16792: 9066 Error 4 opening dom system in 00000170613903D0

2026-06-18 01:00:20.535 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170635D1EF0 [00000000000019ad] { gipcReceiveRequest : peerName 'tcp', data 000001706260DD38, len 48, olen 48, off 0, parentEndp 0000017062951B90, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062951B90 [000000000000199b] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57692)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61013)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 0000017062F209C0, sendp 0000017062F14D90 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x0 }
2026-06-18 01:00:20.558 :  OCRRAW:16792: kgfnConnect2: kgfnGetBeqData failed

2026-06-18 01:00:20.558*:kgfn.c@5129: kgfnConnect2: kgfnGetBeqData failed
2026-06-18 01:00:20.659 :  OCRRAW:16792: kgfnConnect2Int: cstr=(DESCRIPTION=(TCP_USER_TIMEOUT=1)(TRANSPORT_CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=193.169.100.10)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))

2026-06-18 01:00:20.659*:kgfn.c@6929: kgfnConnect2Int: cstr=(DESCRIPTION=(TCP_USER_TIMEOUT=1)(TRANSPORT_CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=193.169.100.10)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))
2026-06-18 01:00:21.081 : CRSMAIN:17996: [     INFO]  Process environment is not initialized yet!
2026-06-18 01:00:22.085 : CRSMAIN:17996: [     INFO]  Process environment is not initialized yet!
2026-06-18 01:00:22.085*:kgfn.c@7107: kgfnConnect2Int: OCISessionBegin failed
2026-06-18 01:00:22.089 :  OCRRAW:16792: kgfnRecordErr 1017 OCI error:
ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:22.089*:kgfn.c@1784: kgfnRecordErrPriv: 1017 error=ORA-01017: invalid username/password; logon denied

2026-06-18 01:00:22.093 : default:16792: clsCredDomClose: Credctx deleted 0000017062DC7EC0
2026-06-18 01:00:22.093 :  OCRRAW:16792: kgfnConnect2: failed to connect

2026-06-18 01:00:22.093*:kgfn.c@5461: kgfnConnect2: failed to connect
2026-06-18 01:00:22.093 :  OCRRAW:16792: kgfnConnect2Retry: failed to connect connect after 1 attempts, 217s elapsed

2026-06-18 01:00:22.093 :  OCRRAW:16792: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:22.093*:kgfo.c@1003: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied

2026-06-18 01:00:22.107 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 00000170635D1680 [0000000000001b73] { gipcReceiveRequest : peerName 'tcp', data 000001706260D9F8, len 48, olen 48, off 0, parentEndp 0000017062951B90, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062951B90 [0000000000001b61] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57712)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61013)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 0000017062F20950, sendp 0000017062FB0770 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x0 }
2026-06-18 01:00:22.131 :  OCRRAW:16792: kgfnFindLocalNode03: kgfn_find_node_sid found no members

2026-06-18 01:00:22.131*:kgfn.c@1412: kgfnFindLocalNode: found no members
2026-06-18 01:00:22.131 :  OCRRAW:16792: kgfnFindLocalNode: not ok

2026-06-18 01:00:22.131*:kgfn.c@1466: kgfnFindLocalNode: not ok
2026-06-18 01:00:22.131 :  OCRRAW:16792: kgfnTgtInit: local node not found, free kgfnpds

2026-06-18 01:00:22.131*:kgfn.c@2252: kgfnTgtInit: not found
2026-06-18 01:00:22.131 :  OCRRAW:16792: kgfnGetBeqData failed init target; inst=(null) flags=0x6000

2026-06-18 01:00:22.131*:kgfn.c@5941: kgfnGetBeqData: kgfnTgtInit failed, inst=NULL flags=0x6000
2026-06-18 01:00:22.133 :   CLSNS:16792: clsns_SetTraceLevel:trace level set to 1.
2026-06-18 01:00:22.385 : default:16792: Inited LSF context: 0000017062DB55B0 
2026-06-18 01:00:22.588 : CLSCRED:16792: clsCredCommonInit: Inited singleton credctx.
2026-06-18 01:00:22.588 : CLSCRED:16792: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2026-06-18 01:00:22.661 :  OCRRAW:16792: 9348 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2026-06-18 01:00:22.673 :  OCRRAW:16792: 9348 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2026-06-18 01:00:22.787 : CLSCRED:16792: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.b6f94c9a7d4f5fe9ff504135765d81e7.system not found
2026-06-18 01:00:22.788 :  OCRRAW:16792: 9066 Error 4 opening dom system in 0000017061390ED0

2026-06-18 01:00:22.815 : GIPCPKT:16792:  gipcmodPacketConsumeRecv: [packet]  Completed receive req 0000017063516B00 [000000000000215e] { gipcReceiveRequest : peerName 'tcp', data 000001706260D9F8, len 48, olen 48, off 0, parentEndp 0000017062951B90, ret gipcretSuccess (0), objFlags 0x0, reqFlags 0x2 }, endp = 0000017062951B90 [000000000000214c] { gipcEndpoint : localAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=57724)(GIPCID=00000000-00000000-0))', remoteAddr 'clsc://(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=61013)(GIPCID=00000000-00000000-0))', numPend 5, numReady 2, numDone 0, numDead 0, numTransfer 0, objFlags 0x0, pidPeer 54456, readyRef 0000000000000000, ready 0, wobj 0000017062F209C0, sendp 00000170635023C0 status 13flags 0x20082712, flags-2 0x100, usrFlags 0x0 }
2026-06-18 01:00:22.875 :  OCRRAW:16792: kgfnConnect2: kgfnGetBeqData failed

2026-06-18 01:00:22.875*:kgfn.c@5129: kgfnConnect2: kgfnGetBeqData failed
2026-06-18 01:00:22.984 :  OCRRAW:16792: kgfnConnect2Int: cstr=(DESCRIPTION=(TCP_USER_TIMEOUT=1)(TRANSPORT_CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=193.169.100.10)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))

2026-06-18 01:00:22.984*:kgfn.c@6929: kgfnConnect2Int: cstr=(DESCRIPTION=(TCP_USER_TIMEOUT=1)(TRANSPORT_CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=193.169.100.10)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))
2026-06-18 01:00:23.088 : CRSMAIN:17996: [     INFO]  Process environment is not initialized yet!
2026-06-18 01:00:24.091 : CRSMAIN:17996: [     INFO]  Process environment is not initialized yet!
2026-06-18 01:00:24.091*:kgfn.c@7107: kgfnConnect2Int: OCISessionBegin failed
2026-06-18 01:00:24.385 :  OCRRAW:16792: kgfnRecordErr 1017 OCI error:
ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:24.385*:kgfn.c@1784: kgfnRecordErrPriv: 1017 error=ORA-01017: invalid username/password; logon denied

2026-06-18 01:00:24.390 : default:16792: clsCredDomClose: Credctx deleted 00000170625365B0
2026-06-18 01:00:24.391 :  OCRRAW:16792: kgfnConnect2: failed to connect

2026-06-18 01:00:24.391*:kgfn.c@5461: kgfnConnect2: failed to connect
2026-06-18 01:00:24.391 :  OCRRAW:16792: kgfnConnect2Retry: failed to connect connect after 1 attempts, 230s elapsed

2026-06-18 01:00:24.391 :  OCRRAW:16792: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:24.391*:kgfo.c@1003: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied

2026-06-18 01:00:24.391 :  OCRRAW:16792: -- trace dump on error exit --

2026-06-18 01:00:24.391 :  OCRRAW:16792: Error [kgfoAl06] in [kgfokge] at kgfo.c:3169

2026-06-18 01:00:24.391 :  OCRRAW:16792: ORA-01017: invalid username/password; logon denied


2026-06-18 01:00:24.391 :  OCRRAW:16792: Category: 7

2026-06-18 01:00:24.391 :  OCRRAW:16792: DepInfo: 1017

2026-06-18 01:00:24.391 :  OCRRAW:16792: -- trace dump end --

  OCRASM:16792: SLOS : SLOS: cat=7, opn=kgfoAl06, dep=1017, loc=kgfokge

2026-06-18 01:00:24.391 :  OCRASM:16792: ASM Error Stack : ORA-01017: invalid username/password; logon denied

2026-06-18 01:00:24.391 :  OCRASM:16792: proprasmo: kgfoCheckMount returned [7]
2026-06-18 01:00:24.391 :  OCRASM:16792: proprasmo: The ASM instance is down
2026-06-18 01:00:24.393 :  OCRRAW:16792: proprioo: Failed to open [+OCR/zxrdsrv/OCRFILE/registry.255.1174550407]. Returned proprasmo() with [26]. Marking location as UNAVAILABLE.
2026-06-18 01:00:24.393 :  OCRRAW:16792: proprioo: No OCR/OLR devices are usable
  OCRUTL:16792: u_fill_errorbuf: Error Info : [仲裁磁盘不足, 无法打开 OCR 设备]
 default:16792: u_set_gbl_comp_error: comptype '107' : error '0'
2026-06-18 01:00:24.394 : default:16792: clsvactversion:4: Retrieving Active Version from local storage.
2026-06-18 01:00:24.399 :  OCRRAW:16792: proprrepauto: The local OCR configuration matches with the configuration published by OCR Cache Writer. No repair required.
2026-06-18 01:00:24.399 :  OCRRAW:16792: proprinit: Could not open raw device 
2026-06-18 01:00:24.418 :  OCRAPI:16792: a_init:16!: Backend init unsuccessful : [26]
2026-06-18 01:00:24.419 :  OCRAPI:16792: estack 'PROC-00026: 访问物理存储时出错'
2026-06-18 01:00:24.422 :  CRSOCR:16792: [    ERROR]  OCR context init failure.  Error: PROC-26: 访问物理存储时出错 存储层错误 [仲裁磁盘不足, 无法打开 OCR 设备] [0]
2026-06-18 01:00:24.427 :    CRSD:16792: [     NONE]  Created alert : (:CRSD00111:) :  Could not init OCR, error: PROC-26: 访问物理存储时出错 存储层错误 [仲裁磁盘不足, 无法打开 OCR 设备] [0]
2026-06-18 01:00:24.427 :    CRSD:16792: [    ERROR] [PANIC] CRSD exiting: Could not init OCR, code: 26
2026-06-18 01:00:24.427 :    CRSD:16792: [     INFO]  Done.

2.1 核心报错堆栈

2026-06-18 01:00:17.638 :  OCRRAW:16792: kgfnRecordErr 1017 OCI error:
ORA-01017: invalid username/password; logon denied

2026-06-18 01:00:24.393 :  OCRRAW:16792: proprioo: No OCR/OLR devices are usable
OCRUTL:16792: u_fill_errorbuf: Error Info : [仲裁磁盘不足, 无法打开 OCR 设备]
2026-06-18 01:00:24.427 :    CRSD:16792: [    ERROR] [PANIC] CRSD exiting: Could not init OCR, code: 26

2.2 关键TNS连接串

CRSD启动过程中，主动通过远程TNS连接ASM实例，连接串如下：

(DESCRIPTION=(TCP_USER_TIMEOUT=1)(TRANSPORT_CONNECT_TIMEOUT=60)(EXPIRE_TIME=1)(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=tcp)(HOST=193.169.100.10)(PORT=1525)))(CONNECT_DATA=(SERVICE_NAME=+ASM)))

3. 日志分析

3.1 第一层报错：ORA-01017 用户名密码无效

CRSD进程在初始化OCR资源时，需要远程连接集群ASM实例校验集群配置，但是SYSASM特权账号密码校验失败，直接导致ASM远程连接失败。

补充：19c GI集群CRSD启动依赖ASM存储读写OCR，ASM认证分为两种模式：

  1. 本地OS认证（优先）：节点本地dbs目录存在orapw+ASM<实例号>本地密码文件，直接走操作系统组认证，无需网络监听连接

  2. 远程TNS认证（兜底）：无本地密码文件时，CRSD自动通过本地监听远程连接ASM实例，依赖ASM共享磁盘组内的密码文件

3.2 第二层报错：ASM实例判定为DOWN，OCR设备全部不可用

proprasmo: The ASM instance is down
proprioo: No OCR/OLR devices are usable

远程ASM连接失败后，集群判定ASM实例未就绪，无法挂载+OCR磁盘组，进而无法读取OCR配置文件、表决磁盘信息，集群仲裁机制失效。

3.3 第三层报错：PROC-26 访问物理存储出错，CRSD直接PANIC

OCR初始化失败是CRSD进程核心前置依赖，前置依赖缺失，集群守护进程无降级运行机制，直接触发PANIC崩溃退出，集群彻底无法启动。

3.4 核心启动死循环（本次故障核心闭环）

19c Windows RAC特有死锁逻辑

  1. CRSD启动必须加载OCR配置，OCR文件存放在共享ASM磁盘组+OCR中

  2. 访问+OCR磁盘组，必须先成功连接并挂载ASM实例

  3. 故障节点无本地ASM兜底密码文件，只能走远程TNS连接ASM

  4. 远程ASM认证依赖共享磁盘组内的密码文件+OCR/orapwASM

  5. ASM未挂载→无法读取共享密码文件→密码校验失败ORA-01017

  6. ASM连接失败→OCR无法访问→CRSD初始化失败→进程PANIC退出

3.5 双节点差异化故障根因

节点名称 本地密码文件状态 认证方式 运行状态
正常节点 存在orapw+ASM1本地兜底密码文件 优先本地OS认证，不依赖网络与共享密码文件 集群正常启动，ASM、CRSD运行正常
故障节点zxrdsrv2 缺失orapw+ASM2本地兜底密码文件 强制走远程TNS网络认证，依赖共享磁盘密码文件 ASM连接失败，CRSD PANIC，集群瘫痪

4. 前置信息核查

4.1 查看当前ASM密码文件配置

C:\Users\Administrator\Desktop\oracle>srvctl config asm -detail
ASM 主目录: <CRS home>
口令文件: +DATA/ATS/PASSWORD/pwdats.ora
口令文件备份: +OCR/orapwASM_backup
ASM 监听程序: LISTENER
ASM 已启用。
ASM 实例计数: 3
集群 ASM 监听程序: ASMNET1LSNR_ASM

4.2 查看数据库当前生效密码文件路径

sqlplus / as sysasm

SQL> SELECT file_name FROM V$PASSWORDFILE_INFO;
FILE NAME
----------------------------------------
+DATA/ATS/PASSWORD/pwdats.ora

4.3 ASMcmd查看密码文件软链接关系

C:\Users\Administrator\Desktop\oracle>asmcmd ls -l +DATA/ATS/PASSWORD/pwdats.ora
Type        Redund  Striped  Time             Sys  Name
PASSWORD    UNPROT  COARSE   MAY 10 04:00:00  N    pwdats.ora => +DATA/ASM/PASSWORD/pwdasm.330.1232858205

4.4 查看备份ASM密码文件用户权限

C:\Users\Administrator\Desktop\oracle>asmcmd lspwusr +OCR/orapwASM_backup
Username sysdba sysoper sysasm
SYS      TRUE    TRUE    TRUE

4.5 本地ASM密码用户核查

C:\Users\Administrator\Desktop\oracle>asmcmd lspwusr
Username sysdba sysoper sysasm
SYSTRUETRUE FALSE

巡检结论：当前集群ASM生效密码文件指向异常路径+DATA磁盘组，而非标准+OCR备份密码文件，同时故障节点缺失本地兜底密码文件，双重问题叠加引发故障。

5. 故障修复

5.1 修复前置准备

  1. 保留正常节点集群全部服务运行，单节点修复故障节点，避免双节点同时停集群扩大故障

  2. 确认+OCR/orapwASM_backup备份密码文件完整且权限正常，SYS账号拥有sysasm、sysdba、sysoper全部权限

  3. 关闭故障节点自动集群启动服务，防止修复过程中CRSD反复重启panic

5.2 修改ASM 默认密码

使用grid管理员账号执行以下命令，将ASM密码文件强制切回OCR磁盘组内完好的备份密码文件：

# 修改ASM全局密码文件配置，指向完好备份文件
srvctl modify asm -pwfile +OCR/orapwASM_backup

# 校验修改后配置
srvctl config asm -detail

修改后正确回显：

ASM 主目录: <CRS home>
口令文件: +OCR/orapwASM_backup
口令文件备份: +OCR/orapwASM_backup
ASM 监听程序: LISTENER
ASM 已启用。
ASM 实例计数: 3
集群 ASM 监听程序: ASMNET1LSNR_ASM

5.3 添加本地ASM密码文件

本次故障根源之一是节点缺失本地密码文件，为彻底规避死循环，从正常节点拷贝本地密码文件至故障节点：

  1. 正常节点路径：E:\app\grid\19.3.0\dbs\orapw+ASM1

  2. 拷贝至故障节点同目录，重命名为：orapw+ASM2

  3. 赋予grid用户完全控制权限，保证本地OS认证优先生效

5.4 重启故障节点集群服务，验证恢复

# 停止本地残留集群资源
crsctl stop crs -f

# 清空集群日志缓存（可选，避免旧日志干扰排查）
del E:\app\grid\diag\crs\zxrdsrv2\crs\trace\crsd.trc

# 启动集群
crsctl start crs

# 全集群资源状态校验
crsctl status res -t

5.5 修复结果验证标准

  • CRSD、CSSD、EVMD三大集群核心进程无panic报错，持续稳定运行

  • ASM实例正常挂载+OCR、+DATA磁盘组，无ORA-01017报错

  • OCR设备正常识别，PROC-26错误彻底消失

  • 数据库实例自动启动，业务恢复正常

windowsoracleoracle 19c墨力计划
最后修改时间：2026-06-23 09:33:30
「喜欢这篇文章，您的关注和赞赏是给作者最好的鼓励」
关注作者
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文章的来源（墨天轮），文章链接，文章作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

评论