Oracle 11g RAC Silent Install For NFS

IT小Chen 2022-03-05

841

环境说明：

OS:Redhat 7.5
DB:11.2.0.4.0 RAC
ASM:NFS

磁盘初始化

检查磁盘大小

root@chenj-ch-05:/oradata3#df -h /oradata*
Filesystem                       Size  Used Avail Use% Mounted on
10.1.10.3:/test_ora19c_nfsB1     100G  710M  100G   1% /oradata1
10.1.10.3:/test_ora19c_01_nfsB1   30G   40M   30G   1% /oradata2
10.1.10.3:/test_ora19c_02_nfsB1   30G   38M   30G   1% /oradata3

检查磁盘挂载情况

root@chenj-ch-05:/oradata3# mount 
10.1.10.3:/test_ora19c_nfsB1 on /oradata1 type nfs (rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.1.10.3,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=10.1.10.3)
10.1.10.3:/test_ora19c_01_nfsB1 on /oradata2 type nfs (rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.1.10.3,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=10.1.10.3)
10.1.10.3:/test_ora19c_02_nfsB1 on /oradata3 type nfs (rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.1.10.3,mountvers=3,mountport=635,mountproto=tcp,local_lock=none,addr=10.1.10.3)

创建共享磁盘文件

dd if=/dev/zero of=/oradata1/test_asm/ocr1  bs=1024k count=2048
dd if=/dev/zero of=/oradata1/test_asm/ocr2  bs=1024k count=2048
dd if=/dev/zero of=/oradata1/test_asm/ocr3  bs=1024k count=2048
dd if=/dev/zero of=/oradata1/test_asm/data1  bs=1024k count=10240
dd if=/dev/zero of=/oradata1/test_asm/data2  bs=1024k count=10240
dd if=/dev/zero of=/oradata1/test_asm/data3  bs=1024k count=10240
dd if=/dev/zero of=/oradata1/test_asm/data4  bs=1024k count=10240
dd if=/dev/zero of=/oradata1/test_asm/data5  bs=1024k count=10240

修改权限

chown grid.oinstall /oradata1/test_asm -R
chmod 660 /oradata1/test_asm/*

静默安装grid

grid@chenj-ch-05:/home/grid$nohup sh /oracle/tmp/grid/runInstaller -silent -noconfig -waitForCompletion -ignorePrereq -ignoreInternalDriverError -showProgress -responseFile /home/grid/grid_install.rsp > /home/grid/install_0304.log &

静默安装grid日志

Starting Oracle Universal Installer...


Checking Temp space: must be greater than 120 MB.   Actual 3949 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 16338 MB    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2022-03-04_02-34-22PM. Please wait ...[WARNING] [INS-30011] The SYS password entered does not conform to the Oracle recommended standards.
   CAUSE: Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
   ACTION: Provide a password that conforms to the Oracle recommended standards.
[WARNING] [INS-30011] The ASMSNMP password entered does not conform to the Oracle recommended standards.
   CAUSE: Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
   ACTION: Provide a password that conforms to the Oracle recommended standards.
[WARNING] [INS-32016] The selected Oracle home contains directories or files.
   CAUSE: The selected Oracle home contained directories or files.
   ACTION: To start with an empty Oracle home, either remove its contents or choose another location.
You can find the log of this install session at:
 /oracle/oraINventory/logs/installActions2022-03-04_02-34-22PM.log


Prepare in progress.
..................................................   9% Done.


Prepare successful.


Copy files in progress.
..................................................   15% Done.
..................................................   20% Done.
..................................................   25% Done.
..................................................   30% Done.
..................................................   35% Done.
..................................................   40% Done.
..................................................   45% Done.
........................................
Copy files successful.


Link binaries in progress.


Link binaries successful.
..................................................   62% Done.


Setup files in progress.


Setup files successful.
..................................................   76% Done.


Perform remote operations in progress.
..................................................   89% Done.


Perform remote operations successful.
The installation of Oracle Grid Infrastructure 11g was successful.
Please check '/oracle/oraINventory/logs/silentInstall2022-03-04_02-34-22PM.log' for more details.
..................................................   94% Done.


Execute Root Scripts in progress.


As a root user, execute the following script(s):
        1. /oracle/oraINventory/orainstRoot.sh
        2. /oracle/grid/product/11.2/root.sh


Execute /oracle/oraINventory/orainstRoot.sh on the following nodes: 
[chenj-ch-05, chenj-ch-06]
Execute /oracle/grid/product/11.2/root.sh on the following nodes: 
[chenj-ch-05, chenj-ch-06]


..................................................   100% Done.


Execute Root Scripts successful.
As install user, execute the following script to complete the configuration.
        1. /oracle/grid/product/11.2/cfgtoollogs/configToolAllCommands RESPONSE_FILE=<response_file>


        Note:
        1. This script must be run on the same host from where installer was run. 
        2. This script needs a small password properties file for configuration assistants that require passwords (refer to install guide documentation).




Successfully Setup Software.

节点1 root.sh执行日志

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /oracle/grid/product/11.2
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /oracle/grid/product/11.2/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding Clusterware entries to inittab
CRS-2672: Attempting to start 'ora.mdnsd' on 'chenj-ch-05'
CRS-2676: Start of 'ora.mdnsd' on 'chenj-ch-05' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'chenj-ch-05'
CRS-2676: Start of 'ora.gpnpd' on 'chenj-ch-05' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'chenj-ch-05'
CRS-2672: Attempting to start 'ora.gipcd' on 'chenj-ch-05'
CRS-2676: Start of 'ora.cssdmonitor' on 'chenj-ch-05' succeeded
CRS-2676: Start of 'ora.gipcd' on 'chenj-ch-05' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'chenj-ch-05'
CRS-2672: Attempting to start 'ora.diskmon' on 'chenj-ch-05'
CRS-2676: Start of 'ora.diskmon' on 'chenj-ch-05' succeeded
CRS-2676: Start of 'ora.cssd' on 'chenj-ch-05' succeeded


ASM created and started successfully.


Disk Group GD_TEST_OCR created successfully.


clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 2c78cf2ab1ac4f99bfbeafadec4caa01.
Successful addition of voting disk 003f25c63a7e4f15bfe6ce115377e75d.
Successful addition of voting disk 083b82cd51cc4fb0bf44ba96af95c99c.
Successfully replaced voting disk group with +GD_TEST_OCR.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   2c78cf2ab1ac4f99bfbeafadec4caa01 (/oradata1/test_asm/ocr1) [GD_TEST_OCR]
 2. ONLINE   003f25c63a7e4f15bfe6ce115377e75d (/oradata1/test_asm/ocr2) [GD_TEST_OCR]
 3. ONLINE   083b82cd51cc4fb0bf44ba96af95c99c (/oradata1/test_asm/ocr3) [GD_TEST_OCR]
Located 3 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'chenj-ch-05'
CRS-2676: Start of 'ora.asm' on 'chenj-ch-05' succeeded
CRS-2672: Attempting to start 'ora.GD_TEST_OCR.dg' on 'chenj-ch-05'
CRS-2676: Start of 'ora.GD_TEST_OCR.dg' on 'chenj-ch-05' succeeded
FirstNode configuration failed at /oracle/grid/product/11.2/crs/install/crsconfig_lib.pm line 9379.
/oracle/grid/product/11.2/perl/bin/perl -I/oracle/grid/product/11.2/perl/lib -I/oracle/grid/product/11.2/crs/install /oracle/grid/product/11.2/crs/install/rootcrs.pl execution failed

执行节点2 root.sh

完整日志：

Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /oracle/grid/product/11.2/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
Adding Clusterware entries to inittab
ohasd failed to start
Failed to start the Clusterware. Last 20 lines of the alert log follow: 
2022-03-04 15:24:05.980: 
[client(55011)]CRS-2101:The OLR was formatted using version 3.


CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node chenj-ch-05, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster


Configure Oracle Grid Infrastructure for a Cluster ... succeeded

查看crs

grid@chenj-ch-06:/home/grid$crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.GD_TEST_OCR.dg
               ONLINE  ONLINE       chenj-ch-05                                  
               ONLINE  ONLINE       chenj-ch-06                                  
ora.asm
               ONLINE  ONLINE       chenj-ch-05              Started             
               ONLINE  ONLINE       chenj-ch-06              Started             
ora.gsd
               OFFLINE OFFLINE      chenj-ch-05                                  
               OFFLINE OFFLINE      chenj-ch-06                                  
ora.net1.network
               ONLINE  ONLINE       chenj-ch-05                                  
               ONLINE  ONLINE       chenj-ch-06                                  
ora.ons
               ONLINE  ONLINE       chenj-ch-05                                  
               ONLINE  ONLINE       chenj-ch-06                                  
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       chenj-ch-05                                  
ora.cvu
      1        OFFLINE OFFLINE                                                   
ora.chenj-ch-05.vip
      1        ONLINE  ONLINE       chenj-ch-05                                  
ora.chenj-ch-06.vip
      1        ONLINE  ONLINE       chenj-ch-06                                  
ora.oc4j
      1        ONLINE  ONLINE       chenj-ch-06                                  
ora.scan1.vip
      1        ONLINE  ONLINE       chenj-ch-05

节点1：执行configToolAllCommands

生成rsp文件

su - grid
vi /home/grid/cfgrsp.properties
oracle.assistants.asm|S_ASMPASSWORD=oracle
oracle.assistants.asm|S_ASMMONITORPASSWORD=oracle
oracle.crs|S_BMCPASSWORD=oracle

开始执行

/oracle/ggrid@chenj-ch-05:/home/grid$/oracle/grid/product/11.2/cfgtoollogs/configToolAllCommands RESPONSE_FILE=/home/grid/cfgrsp.properties
Setting the invPtrLoc to /oracle/grid/product/11.2/oraInst.loc


perform - mode is starting for action: configure


Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn invoke
INFO: Executing ASMCA
Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn invoke
INFO: Command /oracle/grid/product/11.2/bin/asmca -silent -postConfigureASM -oui_internal  
Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: ... GenericInternalPlugIn.handleProcess() entered.
Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: ... GenericInternalPlugIn: getting configAssistantParmas.
Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: ... GenericInternalPlugIn: checking secretArguments.
Mar 4, 2022 3:40:46 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: ... GenericInternalPlugIn: starting read loop.
Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: Read: SYS_PASSWORD_PROMPT
Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: Processing: SYS_PASSWORD_PROMPT for argument tag -sysAsmPassword
Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: Read: ASMSNMP_PASSWORD_PROMPT
Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: Processing: ASMSNMP_PASSWORD_PROMPT for argument tag -asmMonitorPassword
Mar 4, 2022 3:40:49 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: End of argument passing to stdin
Mar 4, 2022 3:40:50 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: Read: 
Mar 4, 2022 3:40:50 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: Read: PostConfiguration completed successfully
Mar 4, 2022 3:40:50 PM oracle.install.driver.oui.config.GenericInternalPlugIn handleProcess
INFO: Read: 


perform - mode finished for action: configure


You can see the log file: /oracle/grid/product/11.2/cfgtoollogs/oui/configActions2022-03-04_03-39-48-PM.logrid/product/11.2/cfgtoollogs/configToolAllCommands RESPONSE_FILE=/home/grid/cfgrsp.properties

查看执行日志

实际调用如下命令：

/oracle/grid/product/11.2/oui/bin/runInstaller -nowait -noconsole -waitforcompletion -ignoreSysPrereqs -updateNodeList -silent CRS=true "CLUSTER_NODES={chenj-ch-05,chenj-ch-06}" ORACLE_HOME=/oracle/grid/product/11.2

日志：

vi /oracle/grid/product/11.2/cfgtoollogs/oui/configActions2022-03-04_03-39-48-PM.logrid/product/11.2/cfgtoollogs/configToolAllCommands RESPONSE_FILE=/home/grid/cfgrsp.properties
###################################################
The action configuration is performing
------------------------------------------------------
The plug-in Update Inventory is running




/oracle/grid/product/11.2/oui/bin/runInstaller -nowait -noconsole -waitforcompletion -ignoreSysPrereqs -updateNodeList -silent CRS=true "CLUSTER_NODES={chenj-ch-05,chenj-ch-06}" ORACLE_HOME=/oracle/grid/product/11.2
Starting Oracle Universal Installer...


Checking swap space: must be greater than 500 MB.   Actual 16338 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /oracle/oraINventory


The plug-in Update Inventory has successfully been performed
------------------------------------------------------
------------------------------------------------------
The plug-in Oracle Net Configuration Assistant is running




Parsing command line arguments:
    Parameter "orahome" = /oracle/grid/product/11.2
    Parameter "orahnam" = Ora11g_gridinfrahome1
    Parameter "instype" = typical
    Parameter "inscomp" = client,oraclenet,javavm,server,ano
    Parameter "insprtcl" = tcp
    Parameter "cfg" = local
    Parameter "authadp" = NO_VALUE
    Parameter "responsefile" = /oracle/grid/product/11.2/network/install/netca_typ.rsp
    Parameter "silent" = true
    Parameter "silent" = true
Done parsing command line arguments.
Oracle Net Services Configuration:
Profile configuration complete.
Profile configuration complete.
chenj-ch-05...
chenj-ch-06...
Oracle Net Listener Startup:
    Listener started successfully.
Listener configuration complete
......
Post-check for cluster services setup was unsuccessful on all the nodes.


The plug-in Oracle Cluster Verification Utility has failed its perform method
------------------------------------------------------
The action configuration has failed its perform method
###################################################

Oracle软件安装

root@chenj-ch-05:/oracle#cp -r oraINventory oraINventory_bak

静默安装

nohup sh /oracle/tmp/database/runInstaller -silent -noconfig -waitForCompletion -ignorePrereq -ignoreInternalDriverError -showProgress -responseFile /home/oracle/rsp/db_install.rsp > /home/oracle/install_0304.log &

安装日志

Starting Oracle Universal Installer...


Checking Temp space: must be greater than 120 MB.   Actual 3949 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 16338 MB    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2022-03-04_03-47-30PM. Please wait ...


You can find the log of this install session at:
 /oracle/oraINventory/logs/installActions2022-03-04_03-47-30PM.log


Prepare in progress.
..................................................   9% Done.


Prepare successful.


Copy files in progress.
..................................................   15% Done.
..................................................   20% Done.
..................................................   25% Done.
..................................................   30% Done.
..................................................   36% Done.
..................................................   41% Done.
..................................................   46% Done.
..................................................   51% Done.
..................................................   56% Done.


Copy files successful.
..........
Link binaries in progress.


Link binaries successful.
..................................................   77% Done.


Setup files in progress.
..................................................   94% Done.


Setup files successful.
The installation of Oracle Database 11g was successful.
Please check '/oracle/oraINventory/logs/silentInstall2022-03-04_03-47-30PM.log' for more details.


Execute Root Scripts in progress.


As a root user, execute the following script(s):
        1. /oracle/db/product/11.2/root.sh


Execute /oracle/db/product/11.2/root.sh on the following nodes: 
[chenj-ch-05, chenj-ch-06]


..................................................   100% Done.


Execute Root Scripts successful.
Successfully Setup Software.

执行脚本

root@chenj-ch-05:/root#/oracle/db/product/11.2/root.sh
Check /oracle/db/product/11.2/install/root_chenj-ch-05_2022-03-04_15-56-21.log for the output of root script
root@chenj-ch-05:/root#vi /oracle/db/product/11.2/install/root_chenj-ch-05_2022-03-04_15-56-21.log


root@chenj-ch-06:/oracle#/oracle/db/product/11.2/root.sh
Check /oracle/db/product/11.2/install/root_chenj-ch-06_2022-03-04_15-57-10.log for the output of root script
root@chenj-ch-06:/oracle#vi /oracle/db/product/11.2/install/root_chenj-ch-06_2022-03-04_15-57-10.log

日志如下：

Performing root user operation for Oracle 11g


The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /oracle/db/product/11.2
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
Finished product-specific root actions.

检查安装

oracle@chenj-ch-05:/home/oracle$sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Fri Mar 4 15:55:38 2022
Copyright (c) 1982, 2013, Oracle.  All rights reserved.
Connected to an idle instance.
SQL>

创建磁盘组

SQL> select name,total_mb,free_mb from v$asm_diskgroup;
NAME                             TOTAL_MB    FREE_MB
------------------------------ ---------- ----------
GD_TEST_OCR                          6144       5218


SQL>
CREATE DISKGROUP GD_TEST_DATA
    EXTERNAL REDUNDANCY
    DISK '/oradata1/test_asm/data1','/oradata1/test_asm/data2','/oradata1/test_asm/data3';


SQL> select name,state,type from v$asm_diskgroup;
NAME                           STATE       TYPE
------------------------------ ----------- ------
GD_TEST_OCR                    MOUNTED     NORMAL
GD_TEST_DATA                   MOUNTED     EXTERN

节点2：

SQL> select name,state,type from v$asm_diskgroup;
NAME                           STATE       TYPE
------------------------------ ----------- ------
GD_TEST_OCR                    MOUNTED     NORMAL
GD_TEST_DATA                   DISMOUNTED




SQL> alter diskgroup GD_TEST_DATA mount;
Diskgroup altered.


SQL> select name,state,type from v$asm_diskgroup;
NAME                           STATE       TYPE
------------------------------ ----------- ------
GD_TEST_OCR                    MOUNTED     NORMAL
GD_TEST_DATA                   MOUNTED     EXTERN

DBCA建库

oracle@chenj-ch-05:/home/oracle/rsp$dbca -silent -createdatabase -responseFile  /home/oracle/rsp/dbca.rsp
日志如下：
Cleaning up failed steps
4% complete
Copying database files
5% complete
6% complete
7% complete
13% complete
19% complete
24% complete
30% complete
33% complete
Creating and starting Oracle instance
35% complete
39% complete
43% complete
46% complete
47% complete
48% complete
50% complete
52% complete
Creating cluster database views
54% complete
71% complete
Completing Database Creation
74% complete
77% complete
85% complete
94% complete
100% complete
Look at the log file "/oracle/db/cfgtoollogs/dbca/test/test5.log" for further details.

RAC检查

oracle@chenj-ch-05:/home/oracle/rsp$srvctl config database
test


oracle@chenj-ch-05:/home/oracle/rsp$srvctl status database -d test
Instance test1 is running on node chenj-ch-05
Instance test2 is running on node chenj-ch-06


grid@chenj-ch-05:/home/grid$crsctl stat res -t


--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.GD_TEST_DATA.dg
               ONLINE  ONLINE       chenj-ch-05                                  
               ONLINE  ONLINE       chenj-ch-06                                  
ora.GD_TEST_OCR.dg
               ONLINE  ONLINE       chenj-ch-05                                  
               ONLINE  ONLINE       chenj-ch-06                                  
ora.LISTENER.lsnr
               ONLINE  ONLINE       chenj-ch-05                                  
               ONLINE  ONLINE       chenj-ch-06                                  
ora.asm
               ONLINE  ONLINE       chenj-ch-05              Started             
               ONLINE  ONLINE       chenj-ch-06              Started             
ora.gsd
               OFFLINE OFFLINE      chenj-ch-05                                  
               OFFLINE OFFLINE      chenj-ch-06                                  
ora.net1.network
               ONLINE  ONLINE       chenj-ch-05                                  
               ONLINE  ONLINE       chenj-ch-06                                  
ora.ons
               ONLINE  ONLINE       chenj-ch-05                                  
               ONLINE  ONLINE       chenj-ch-06                                  
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       chenj-ch-05                                  
ora.cvu
      1        OFFLINE OFFLINE                                                   
ora.chenj-ch-05.vip
      1        ONLINE  ONLINE       chenj-ch-05                                  
ora.chenj-ch-06.vip
      1        ONLINE  ONLINE       chenj-ch-06                                  
ora.oc4j
      1        ONLINE  ONLINE       chenj-ch-06                                  
ora.scan1.vip
      1        ONLINE  ONLINE       chenj-ch-05                                  
ora.test.db
      1        ONLINE  ONLINE       chenj-ch-05              Open                
      2        ONLINE  ONLINE       chenj-ch-06              Open

数据测试

节点1：

SQL> create user cjc identified by a;
User created.


SQL> grant dba to cjc;
Grant succeeded.


SQL> conn cjc/a
Connected.
SQL> 
SQL> 
SQL> create table t1 as select level as id from dual connect by level <=10;
Table created.


SQL> select * from t1;
        ID
----------
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
10 rows selected.

节点2：

oracle@chenj-ch-06:/home/oracle$sqlplus cjc/a
SQL*Plus: Release 11.2.0.4.0 Production on Fri Mar 4 16:17:40 2022
Copyright (c) 1982, 2013, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options


SQL> select * from t1;
        ID
----------
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
10 rows selected.

重启：

oracle@chenj-ch-05:/home/oracle$srvctl stop database -d test
oracle@chenj-ch-05:/home/oracle$srvctl start database -d test

重启crs

停crs

root@chenj-ch-06:/oracle/grid/product/11.2/bin#./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.crsd' on 'chenj-ch-06'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.GD_TEST_OCR.dg' on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.test.db' on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.oc4j' on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.chenj-ch-06.vip' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.test.db' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.GD_TEST_DATA.dg' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.chenj-ch-06.vip' on 'chenj-ch-06' succeeded
CRS-2672: Attempting to start 'ora.chenj-ch-06.vip' on 'chenj-ch-05'
CRS-2677: Stop of 'ora.GD_TEST_DATA.dg' on 'chenj-ch-06' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'chenj-ch-06' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'chenj-ch-05'
CRS-2677: Stop of 'ora.oc4j' on 'chenj-ch-06' succeeded
CRS-2672: Attempting to start 'ora.oc4j' on 'chenj-ch-05'
CRS-2676: Start of 'ora.chenj-ch-06.vip' on 'chenj-ch-05' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'chenj-ch-05' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'chenj-ch-05'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'chenj-ch-05' succeeded
CRS-2677: Stop of 'ora.GD_TEST_OCR.dg' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.asm' on 'chenj-ch-06' succeeded
CRS-2674: Start of 'ora.oc4j' on 'chenj-ch-05' failed
CRS-2679: Attempting to clean 'ora.oc4j' on 'chenj-ch-05'
CRS-2681: Clean of 'ora.oc4j' on 'chenj-ch-05' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.ons' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.net1.network' on 'chenj-ch-06' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'chenj-ch-06' has completed
CRS-2677: Stop of 'ora.crsd' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.crf' on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.ctssd' on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.evmd' on 'chenj-ch-06'
CRS-2673: Attempting to stop 'ora.asm' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.mdnsd' on 'chenj-ch-06' succeeded
CRS-2677: Stop of 'ora.evmd' on 'chenj-ch-06' succeeded
CRS-2677: Stop of 'ora.crf' on 'chenj-ch-06' succeeded
CRS-2677: Stop of 'ora.asm' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'chenj-ch-06' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.cssd' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.gipcd' on 'chenj-ch-06' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'chenj-ch-06'
CRS-2677: Stop of 'ora.gpnpd' on 'chenj-ch-06' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'chenj-ch-06' has completed
CRS-4133: Oracle High Availability Services has been stopped.

启动crs过程

root@chenj-ch-06:/oracle/grid/product/11.2/bin#./crsctl start crs
2022-03-04 16:50:50.088: 
[ohasd(69993)]CRS-2112:The OLR service started on node chenj-ch-06.
2022-03-04 16:50:50.093: 
[ohasd(69993)]CRS-1301:Oracle High Availability Service started on node chenj-ch-06.
2022-03-04 16:50:50.093: 
[ohasd(69993)]CRS-8017:location: /etc/oracle/lastgasp has 2 reboot advisory log files, 0 were announced and 0 errors occurred
2022-03-04 16:50:53.370: 
[/oracle/grid/product/11.2/bin/orarootagent.bin(70033)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running). 
2022-03-04 16:50:57.737: 
[gpnpd(70128)]CRS-2328:GPNPD started on node chenj-ch-06. 
2022-03-04 16:51:00.056: 
[cssd(70196)]CRS-1713:CSSD daemon is started in clustered mode
2022-03-04 16:51:01.902: 
[ohasd(69993)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2022-03-04 16:51:01.902: 
[ohasd(69993)]CRS-2769:Unable to failover resource 'ora.diskmon'.
2022-03-04 16:51:20.542: 
[cssd(70196)]CRS-1707:Lease acquisition for node chenj-ch-06 number 2 completed
2022-03-04 16:51:21.814: 
[cssd(70196)]CRS-1605:CSSD voting file is online: /oradata1/test_asm/ocr1; details in /oracle/grid/product/11.2/log/chenj-ch-06/cssd/ocssd.log.
2022-03-04 16:51:21.858: 
[cssd(70196)]CRS-1605:CSSD voting file is online: /oradata1/test_asm/ocr2; details in /oracle/grid/product/11.2/log/chenj-ch-06/cssd/ocssd.log.
2022-03-04 16:51:21.898: 
[cssd(70196)]CRS-1605:CSSD voting file is online: /oradata1/test_asm/ocr3; details in /oracle/grid/product/11.2/log/chenj-ch-06/cssd/ocssd.log.
2022-03-04 16:51:25.529: 
[cssd(70196)]CRS-1601:CSSD Reconfiguration complete. Active nodes are chenj-ch-05 chenj-ch-06 .
2022-03-04 16:51:27.063: 
[ctssd(70307)]CRS-2403:The Cluster Time Synchronization Service on host chenj-ch-06 is in observer mode.
2022-03-04 16:51:27.309: 
[ctssd(70307)]CRS-2407:The new Cluster Time Synchronization Service reference node is host chenj-ch-05.
2022-03-04 16:51:27.310: 
[ctssd(70307)]CRS-2401:The Cluster Time Synchronization Service started on host chenj-ch-06.
2022-03-04 16:51:29.059: 
[ohasd(69993)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2022-03-04 16:51:29.059: 
[ohasd(69993)]CRS-2769:Unable to failover resource 'ora.diskmon'.
2022-03-04 16:51:47.564: 
[crsd(70414)]CRS-1012:The OCR service started on node chenj-ch-06.
2022-03-04 16:51:47.572: 
[evmd(70329)]CRS-1401:EVMD started on node chenj-ch-06.
2022-03-04 16:51:49.614: 
[crsd(70414)]CRS-1201:CRSD started on node chenj-ch-06.
2022-03-04 16:51:57.716: 
[client(70701)]CRS-4743:File /oracle/grid/product/11.2/oc4j/j2ee/home/OC4J_DBWLM_config/system-jazn-data.xml was updated from OCR(Size: 13365(New), 13378(Old) bytes)

查看会话

SQL> select inst_id,count(*) from gv$session group by inst_id;
   INST_ID   COUNT(*)
---------- ----------
         1         41
         2         39

排错过程

问题一：crsconfig_lib.pm line 9379

报错：

FirstNode configuration failed at /oracle/grid/product/11.2/crs/install/crsconfig_lib.pm line 9379.
/oracle/grid/product/11.2/perl/bin/perl -I/oracle/grid/product/11.2/perl/lib -I/oracle/grid/product/11.2/crs/install /oracle/grid/product/11.2/crs/install/rootcrs.pl execution failed

手动执行：

/oracle/grid/product/11.2/perl/bin/perl -I/oracle/grid/product/11.2/perl/lib -I/oracle/grid/product/11.2/crs/install /oracle/grid/product/11.2/crs/install/rootcrs.pl
...
PRCR-1079 : Failed to start resource ora.LISTENER_SCAN1.lsnr
CRS-5702: Resource 'ora.LISTENER_SCAN1.lsnr' is already running on 'chenj-ch-05'
FirstNode configuration failed at /oracle/grid/product/11.2/crs/install/crsconfig_lib.pm line 9379.

检查crsd.log

grid@chenj-ch-05:/oracle/grid/product/11.2/log/chenj-ch-05/crsd$vi crsd.log 
2022-03-04 15:06:08.395: [UiServer][1295947520]{1:56663:491} Container [ Name: ORDER
        MESSAGE:
        TextMessage[CRS-2674: Start of 'ora.oc4j' on 'chenj-ch-05' failed]
        MSGTYPE:
        TextMessage[1]
        OBJID:
        TextMessage[ora.oc4j]
        WAIT:
        TextMessage[0]
]


...
2022-03-04 15:11:15.902: [   CRSPE][1298048768]{1:56663:598} Op 0x7fddfc1266d0 has 3 WOs
2022-03-04 15:11:15.903: [   CRSPE][1298048768]{1:56663:598} ICE has queued an operation. Details: Operation [START of [ora.oc4j 1 1] on [chenj-ch-05] : local=0, unplanned=00x7fddfc1266d0] cannot run cause it needs W lock for: WO for Placement Path RI:[ora.oc4j 1 1] server [chenj-ch-05] target states [ONLINE INTERMEDIATE ], locked by op [START of [ora.oc4j 1 1] on [chenj-ch-05] : local=0, unplanned=00x7fddfc123e60]. Owner: CRS-2682: It is locked by 'grid' for command 'Start Resource' issued from 'chenj-ch-05'

手动启动

grid@chenj-ch-05:/oracle/grid/product/11.2/log/chenj-ch-05/crsd$srvctl start oc4j -v
OC4J could not be started
PRCR-1079 : Failed to start resource ora.oc4j
CRS-2674: Start of 'ora.oc4j' on 'chenj-ch-05' failed
CRS-2632: There are no more servers to try to place resource 'ora.oc4j' on that would satisfy its placement policy
先跳过，不处理，ora.oc4不影响安装

问题二:gpnptool rc=256

节点1执行root.sh遇到的另一个错误：

错误：

Failed to sign a peer profile for Oracle Cluster GPnP. gpnptool rc=256
Creation of Oracle GPnP peer profile failed for chenj-ch-05
Failed to create GPnP peer profile for chenj-ch-05 at /oracle/grid/product/11.2/crs/install/crsconfig_lib.pm line 5013.
/oracle/grid/product/11.2/perl/bin/perl -I/oracle/grid/product/11.2/perl/lib -I/oracle/grid/product/11.2/crs/install /oracle/grid/product/11.2/crs/install/rootcrs.pl execution failed

问题原因：

在安装grid阶段，执行root.sh前，更新了31537677 Opatch，导致root.sh执行失败。

回退OPatch

opatch rollback -id 31537677

问题三：CRS-2101

执行节点2 root.sh

日志：

报错，hang住

ohasd failed to start
Failed to start the Clusterware. Last 20 lines of the alert log follow: 
2022-03-04 15:24:05.980: 
[client(55011)]CRS-2101:The OLR was formatted using version 3.

解决方案：

root@chenj-ch-06:/root#cd /var/tmp/.oracle/
root@chenj-ch-06:/var/tmp/.oracle#ls
npohasd
root@chenj-ch-06:/var/tmp/.oracle#ll -rth
total 0
prw-r--r-- 1 root root 0 Mar  3 21:41 npohasd
root@chenj-ch-06:/var/tmp/.oracle#rm npohasd 
rm: remove fifo ‘npohasd’? y
root@chenj-ch-06:/var/tmp/.oracle#touch npohasd
root@chenj-ch-06:/var/tmp/.oracle#chmod 755 npohasd 
root@chenj-ch-06:/var/tmp/.oracle#ll -rth
total 0
-rwxr-xr-x 1 root root 0 Mar  3 21:48 npohasd

问题四:Oracle Grid Infrastructure is not installed

###执行configToolAllCommands

必须执行configToolAllCommands，如果不执行，后面安装oracle时，会提示下面错误：

 CAUSE: Before you can install Oracle RAC, you must install Oracle Grid Infrastructure on all servers (Oracle Clusterware and Oracle ASM) to create a cluster.
 ACTION: Oracle Grid Infrastructure is not installed. Install it either from the separate installation media included in your media pack, or install it by downloading it from Electronic Product Delivery (EPD) or the Oracle Technology Network (OTN). Oracle Grid Infrastructure normally is installed by a different operating system user than the one used for Oracle Database. It may need to be installed by your system administrator. See the installation guide for more details.

问题五:ORA-12545

安装oracle软件时遇到如下错误：

执行root.sh时，提示rootadd_rdbms.sh、rootadd_filemap.sh文件找不到，sqlplus登录时提示ORA-12545

Performing root user operation for Oracle 11g


The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /oracle/db/product/11.2
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
/oracle/db/product/11.2/root.sh: line 14: /oracle/db/product/11.2/rdbms/install/rootadd_rdbms.sh: No such file or directory
/oracle/db/product/11.2/root.sh: line 16: /oracle/db/product/11.2/rdbms/install/rootadd_filemap.sh: No such file or directory

sqlplus登录报错ORA-12545

oracle@chenj-ch-05:/home/oracle$source .bash_profile 
oracle@chenj-ch-05:/home/oracle$sqlplus / as sysdba
SQL*Plus: Release 11.2.0.4.0 Production on Fri Mar 4 10:04:57 2022
Copyright (c) 1982, 2013, Oracle.  All rights reserved.
ERROR:


ORA-12545: Connect failed because target host or object does not exist

通过history发现，在安装oracle时，解决软件包只解压了1个，第2个包忘解压了，静默安装时居然没有报错，安装后发现少了很多文件。

卸载Oracle软件，重新安装，具体需要删除$ORACLE_BASE，$ORACLE_HOME，/etc/ora*，/var/tmp/.oracle/*等文件。

#####chenjuchao 20220304 20:50#####

oracle

文章转载自IT小Chen，如果涉嫌侵权，请发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

Oracle 11g RAC Silent Install For NFS

评论