记录一次vip注册信息错误，导致监听启动失败的案例。

故障现象

描述的故障现象是2节点的rac监听无法启动，也无法关闭。同时vip状态为UNKOWN,其他进程状态正常。

[grid@rac1 ~]$ srvctl status listener -n rac1
Listener LISTENER is enabled on node(s): rac1
Listener LISTENER is running on node(s): rac1
2节点监听not running
[grid@rac1 ~]$ srvctl status listener -n rac2
Listener LISTENER is enabled on node(s): rac2
Listener LISTENER is not running on node(s): rac2

ora.rac1.vip
      1        ONLINE  ONLINE       rac1                                   
ora.rac2.vip
      1        ONLINE  UNKNOWN      rac1 

分析过程

listener是依赖于vip的。首先从vip故障入手。尝试手工拉起vip。

[grid@rac1 ~]$ srvctl start vip -i rac2
PRCR-1079 : Failed to start resource ora.rac2.vip
CRS-2680: Clean of 'ora.rac2.vip' on 'rac1' failed
CRS-5802: Unable to start the agent process

报错中提示在rac1上启动ora.rac2.vip，与crsctl stat res -t中的信息正好对的上。

怀疑注册信息有问题。由于刚收到故障信息时，说之前一直是这样没问题，属于突发故障，虽然很诧异，还是没敢肯定这个问题，被误导排查了很久crsd和cssd日志，耽误了一段时间。还是要相信自己的判断的。
以下信息基本可以断定，vip信息有误引起：

• 2节点的vip no exist

[grid@rac1 admin]$ srvctl config vip -n rac2
PRKO-2310 : VIP does not exist on node rac2.

[grid@rac1 admin]$ srvctl config vip -n rac1
VIP exists: /rac1-vip/xxxxxxxx/eth0, hosting node rac1
xxxxxxxx为网络信息
grid@rac1 ~]$ srvctl config nodeapps -n rac1
-n <node_name> option has been deprecated.
Network exists: xxxxxxxx, type static
VIP exists: /rac1-vip/xxxxxxxx/eth0, hosting node rac1
GSD exists
ONS exists: Local port xx, remote port xx, EM port xx

• 2节点没有vip信息

[grid@rac1 ~]$ srvctl config nodeapps -n rac2
-n <node_name> option has been deprecated.
Network exists: xxxxxxxx, type static
GSD exists
ONS exists: Local port xx, remote port xx, EM port xx

• 尝试添加vip

[root@rac1 ~]# $GRID_HOME/bin/srvctl add vip -n rac2 -A rac2-vip/255.255.255.0/eth0 -k 1
PRCR-1086 : resource ora.rac2.vip is already registered
显示已经注册，使用remove删除重新添加
[root@rac1 ~]# $GRID_HOME/bin/srvctl remove vip -i rac2
Please confirm that you intend to remove the VIPs rac2 (y/[n]) y
PRCN-2034 : Could not find dependent resource network for resource ora.rac2.vip:
无法找到依赖的资源，使用-f 强制删除
[root@rac1 ~]#$GRID_HOME/bin/srvctl remove vip -i rac2 -f
PRKO-2381 : VIP rac2 is not removed successfully:  
PRCN-2034 : Could not find dependent resource network for resource ora.rac2.vip: 
PRCN-2034 : Could not find dependent resource network for resource ora.rac2.vip:

不能使用srvctl删除，应该是注册的时候不是使用srvctl注册的。

最终解决方案

$GRID_HOME/crs_unregister ora.rac2.vip
$GRID_HOME/bin/srvctl add vip -n rac2 -A rac2-vip/xxxxxxxx/eth0 -k 1
srvctl start vip -n rac2

• vip信息正常

[grid@rac1 ~]$ srvctl config vip -n rac2
VIP exists: /rac2-vip/xxxxxxxx, hosting node rac2

[grid@rac1 ~]$ srvctl config nodeapps -n rac2
-n <node_name> option has been deprecated.
Network exists: xxxxxxxx, type static
VIP exists: /rac2-vip/xxxxxxxx/eth0, hosting node rac2
GSD exists
ONS exists: Local port xx, remote port xx, EM port xx

vip故障处理后，重新启动监听恢复正常。