Centos 7.7下Oracle 11g RAC部署
- 操作系统环境
1.1linux操作系统配置
两台主机IP规划信息如下：
rac1 rac2
Ip：192.168.198.180 Ip：192.168.198.181
priv:10.10.10.81 priv: 10.10.10.82
vip: 192.168.198.182 vip: 192.168.198.183
scan-ip:192.168.198.184
Linux系统版本均为7.7
Oracle版本为11.2.0.4.0
1.2hosts文件配置
[root@rac1 ~]# vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
#public
192.168.198.180 rac1
192.168.198.181 rac2
#vip
192.168.198.182 rac1-vip
192.168.198.183 rac2-vip
#priv
10.10.10.81 rac1-priv
10.10.10.82 rac2-priv
#scan
192.168.198.184 rac-cluster-scan
两个节点的hosts文件内容要相同
1.3配置内核参数
[root@rac1 ~]# vi /etc/sysctl.conf
sysctl settings are defined through files in
/usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
Vendors settings live in /usr/lib/sysctl.d/.
To override a whole file, create a new file with the same in
/etc/sysctl.d/ and put new settings there. To override
only specific settings, add a file with a lexically later
name in /etc/sysctl.d/ and put new settings there.
For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 8389934592 # kernel.shmmax要以自己实际内存大小而定
kernel.shmall = 268435456
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
1.4关闭防火墙
[root@rac1 ~]# systemctl stop firewalld
[root@rac1 ~]# systemctl disable firewalld
[root@rac1 ~]# systemctl status firewalld
1.5 关闭SELINUX
[root@rac1 ~]# vi /etc/selinux/config
This file controls the state of SELinux on the system.
SELINUX= can take one of these three values:
enforcing - SELinux security policy is enforced.
permissive - SELinux prints warnings instead of enforcing.
disabled - No SELinux policy is loaded.
SELINUX=disabled
SELINUXTYPE= can take one of three values:
targeted - Targeted processes are protected,
minimum - Modification of targeted policy. Only selected processes are protected.
mls - Multi Level Security protection.
SELINUXTYPE=targeted
1.6安全限制调整
[root@rac1 ~]# vi /etc/security/limits.conf
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
1.7安装rpm包
[root@rac1 ~]# yum -y install binutils compat-libstdc+±33 gcc gcc-c++ glibc glibc-common glibc-devel ksh libaio libaio-devel libgcc libstdc++ libstdc+±devel make sysstat openssh-clients compat-libcap1 xorg-x11-utils xorg-x11-xauth elfutils unixODBC unixODBC-devel libXp elfutils-libelf elfutils-libelf-devel smartmontools glibc-headers
2.用户配置
2.1添加Oracle/grid用户
[root@rac1 ~]# groupadd -g 54321 oinstall
[root@rac1 ~]# groupadd -g 54322 dba
[root@rac1 ~]# groupadd -g 54323 oper
[root@rac1 ~]# groupadd -g 54324 backupdba
[root@rac1 ~]# groupadd -g 54325 dgdba
[root@rac1 ~]# groupadd -g 54326 kmdba
[root@rac1 ~]# groupadd -g 54327 asmdba
[root@rac1 ~]# groupadd -g 54328 asmoper
[root@rac1 ~]#groupadd -g 54329 asmadmin
[root@rac1 ~]# groupadd -g 54330 racdba
[root@rac1 ~]# useradd -u 54321 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle
[root@rac1 ~]# useradd -u 54322 -g oinstall -G asmadmin,asmdba,asmoper,dba grid
[root@rac1 ~]# passwd oracle
[root@rac1 ~]# passwd grid
2.2创建目录
[root@rac1 ~]# mkdir -p /oracle/app/11.2.0/grid
[root@rac1 ~]# mkdir -p /oracle/app/grid
[root@rac1 ~]# mkdir -p /oracle/app
[root@rac1 ~]# mkdir -p /oracle/app/oracle/product/11.2.0/dbhome_1
[root@rac1 ~]# chown -R oracle:oinstall /oracle
[root@rac1 ~]# chown -R grid:oinstall /oracle/app/11.2.0/grid
[root@rac1 ~]# chown -R grid:oinstall /oracle/app/grid
[root@rac1 ~]# chown -R oracle:oinstall /oracle/app/oracle
[root@rac1 ~]# chmod 771 /oracle/
[root@rac1 ~]# chmod 771 /oracle/app
2.3设置Oracle用户环境变量
[root@rac1 ~]# su – oracle
[oracle@rac1 ~]$ vi .bash_profile
.bash_profile
Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
User specific environment and startup programs
PATH=PATH:PATH:PATH:HOME/.local/bin:$HOME/bin
export PATH
export ORACLE_BASE=/u01/oracle
export ORACLE_HOME=ORACLE_BASE/product/11.2.0/db_1 export ORACLE_SID=beijing1 #节点俩边实例名分别为beijing1和beijing2 export LANG=en_US.UTF-8 export NLS_LANG=american_america.ZHS16GBK export NLS_DATE_FORMAT=“yyyy-mm-dd hh24:mi:ss” export PATH=.:{PATH}:HOME/bin:HOME/bin:HOME/bin:ORACLE_HOME/bin:ORACLEHOME/OPatchexportPATH=ORACLE_HOME/OPatch export PATH=ORACLEHOME/OPatchexportPATH={PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
export PATH=PATH:{PATH}:PATH:ORACLE_BASE/common/oracle/bin:/home/oracle/run
export ORACLE_TERM=xterm
export LD_LIBRARY_PATH=ORACLEHOME/libexportLDLIBRARYPATH=ORACLE_HOME/lib export LD_LIBRARY_PATH=ORACLEHOME/libexportLDLIBRARYPATH={LD_LIBRARY_PATH}:ORACLEHOME/oracm/libexportLDLIBRARYPATH=ORACLE_HOME/oracm/lib export LD_LIBRARY_PATH=ORACLEHOME/oracm/libexportLDLIBRARYPATH={LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export CLASSPATH=ORACLEHOME/JREexportCLASSPATH=ORACLE_HOME/JRE export CLASSPATH=ORACLEHOME/JREexportCLASSPATH={CLASSPATH}:ORACLEHOME/jlibexportCLASSPATH=ORACLE_HOME/jlib export CLASSPATH=ORACLEHOME/jlibexportCLASSPATH={CLASSPATH}:ORACLEHOME/rdbms/jlibexportCLASSPATH=ORACLE_HOME/rdbms/jlib export CLASSPATH=ORACLEHOME/rdbms/jlibexportCLASSPATH={CLASSPATH}:ORACLEHOME/network/jlibexportTHREADSFLAG=nativeexportTEMP=/tmpexportTMPDIR=/tmpexportGIHOME=/oracle/app/11.2.0/gridexportPATH=ORACLE_HOME/network/jlib export THREADS_FLAG=native export TEMP=/tmp export TMPDIR=/tmp export GI_HOME=/oracle/app/11.2.0/grid export PATH=ORACLEHOME/network/jlibexportTHREADSFLAG=nativeexportTEMP=/tmpexportTMPDIR=/tmpexportGIHOME=/oracle/app/11.2.0/gridexportPATH={PATH}:GIHOME/binexportORANLS10=GI_HOME/bin export ORA_NLS10=GIHOME/binexportORANLS10=GI_HOME/nls/data
umask 022
export TMOUT=0
2.4设置grid用户环境变量
[grid@rac1 ~]$ vi .bash_profile
.bash_profile
Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
User specific environment and startup programs
PATH=PATH:PATH:PATH:HOME/.local/bin:HOME/bin export PATH export ORACLE_BASE=/oracle/app/grid export ORACLE_HOME=/oracle/app/11.2.0/grid export ORACLE_SID=+ASM1 #两边分别为+ASM1和+ASM2 export NLS_LANG=american_america.ZHS16GBK export NLS_DATE_FORMAT=“yyyy-mm-dd hh24:mi:ss” export PATH=.:{PATH}:HOME/bin:HOME/bin:HOME/bin:ORACLE_HOME/bin:ORACLEHOME/OPatchexportPATH=ORACLE_HOME/OPatch export PATH=ORACLEHOME/OPatchexportPATH={PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
export PATH=PATH:{PATH}:PATH:ORACLE_BASE/common/oracle/bin
export ORACLE_TERM=xterm
export LD_LIBRARY_PATH=ORACLEHOME/libexportLDLIBRARYPATH=ORACLE_HOME/lib export LD_LIBRARY_PATH=ORACLEHOME/libexportLDLIBRARYPATH={LD_LIBRARY_PATH}:ORACLEHOME/oracm/libexportLDLIBRARYPATH=ORACLE_HOME/oracm/lib export LD_LIBRARY_PATH=ORACLEHOME/oracm/libexportLDLIBRARYPATH={LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export CLASSPATH=ORACLEHOME/JREexportCLASSPATH=ORACLE_HOME/JRE export CLASSPATH=ORACLEHOME/JREexportCLASSPATH={CLASSPATH}:ORACLEHOME/jlibexportCLASSPATH=ORACLE_HOME/jlib export CLASSPATH=ORACLEHOME/jlibexportCLASSPATH={CLASSPATH}:ORACLEHOME/rdbms/jlibexportCLASSPATH=ORACLE_HOME/rdbms/jlib export CLASSPATH=ORACLEHOME/rdbms/jlibexportCLASSPATH={CLASSPATH}:$ORACLE_HOME/network/jlib
export THREADS_FLAG=native
export TEMP=/tmp
export TMPDIR=/tmp
umask 022
export TMOUT=0
~
3. UDEV绑盘
3.1创建4块新的虚拟磁盘
先关闭两边虚机
以此分别创建3个1G，1个10G的磁盘
以自己建的磁盘顺序依次绑定节点
第二台虚机添加硬盘时选择使用现有磁盘
找到刚刚创建的4块磁盘并绑定相同节点
3.2修改虚机参数
找到虚机.vmx参数文件
编辑该文件在文件末尾加上
diskLib.dataCacheMaxSize = “0”
diskLib.dataCacheMaxReadAheadSize = “0”
diskLib.dataCacheMinReadAheadSize = “0”
diskLib.dataCachePageSize = “4096”
diskLib.maxUnsyncedWrites = “0”
disk.locking = “FALSE”
disk.enableUUID = “TRUE”
两边都要添加之后保存文件再开机
3.3绑定磁盘组
[root@rac1 ~]# fdisk –l
查看自己两边磁盘UUID是否相同
[root@node1 ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdb
360000000000000000e00000000020001
[root@node1 ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdc
360000000000000000e00000000020002
[root@node1 ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdd
360000000000000000e00000000020003
[root@node1 ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sde
360000000000000000e00000000020004
[root@node1 ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdf
360000000000000000e00000000020005
两边添加绑定文件
[root@rac1 ~]# vi /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL==“sdb”, SUBSYSTEM==“block”, PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/sdb", RESULT==“360000000000000000e00000000020001”, SYMLINK+=“asm-crs1”, OWNER=“grid”, GROUP=“asmadmin”, MODE=“0660”
KERNEL==“sdc”, SUBSYSTEM==“block”, PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/sdc", RESULT==“360000000000000000e00000000020002”, SYMLINK+=“asm-crs2”, OWNER=“grid”, GROUP=“asmadmin”, MODE=“0660”
KERNEL==“sdd”, SUBSYSTEM==“block”, PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/sdd", RESULT==“360000000000000000e00000000020003”, SYMLINK+=“asm-crs3”, OWNER=“grid”, GROUP=“asmadmin”, MODE=“0660”
KERNEL==“sde”, SUBSYSTEM==“block”, PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/sde", RESULT==“360000000000000000e00000000020004”, SYMLINK+=“asm-data”, OWNER=“grid”, GROUP=“asmadmin”, MODE=“0660”
KERNEL==“sdf”, SUBSYSTEM==“block”, PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/sdf", RESULT==“360000000000000000e00000000020005”, SYMLINK+=“asm-fra”, OWNER=“grid”, GROUP=“asmadmin”, MODE=“0660”
两边编辑绑定文件后，依次执行以下命令
[root@rac1 ~]# systemctl status systemd-udevd.service
[root@rac1 ~]# systemctl enable systemd-udevd.service
[root@rac1 ~]# /usr/sbin/udevadm control --reload-rules
[root@rac1 ~]# /usr/sbin/udevadm trigger --type=devices
[root@rac1 ~]# ls -l /dev/sd*
brw-rw---- 1 root disk 8, 0 Nov 14 11:23 /dev/sda
brw-rw---- 1 root disk 8, 1 Nov 14 11:23 /dev/sda1
brw-rw---- 1 root disk 8, 2 Nov 14 11:23 /dev/sda2
brw-rw---- 1 grid asmadmin 8, 16 Nov 14 11:52 /dev/sdb
brw-rw---- 1 grid asmadmin 8, 32 Nov 14 11:53 /dev/sdc
brw-rw---- 1 grid asmadmin 8, 49 Nov 14 11:56 /dev/sdd
brw-rw---- 1 grid asmadmin 8, 65 Nov 14 11:56 /dev/sde
[root@rac1 ~]# ls -l /dev/asm*
lrwxrwxrwx 1 root root 4 Nov 14 11:57 /dev/asm-crs1 -> sdb
lrwxrwxrwx 1 root root 4 Nov 14 11:56 /dev/asm-crs2 -> sdc
lrwxrwxrwx 1 root root 4 Nov 14 11:56 /dev/asm-crs3 -> sdd
lrwxrwxrwx 1 root root 4 Nov 14 11:56 /dev/asm-data -> sde
4安装集群软件
4.1先上传grid安装包到grid家目录下
[grid@rac1 ~]$ ll
-rwxwxr-x 1 grid oinstall 1205251894 2月 27 2015 p13390677_112040_Linux-x86-64_3of7.zip
上传过来后要修改grid安装包权限
安装GNOME Desktop图形桌面服务
yum groupinstall “GNOME Desktop”
查看系统运行模式
systemctl get-default
切换到桌面运行模式
systemctl set-default graphical.target
启动桌面模式
init 5
VNC配置
yum install tigervnc-server
启动VNC服务
$ vncserver :1
You will require a password to access your desktops.
Password: ##登录vnc的密码，以后可用通过vncpasswd命令更改密码
Verify: ##重新输入登录vnc的密码
Would you like to enter a view-only password (y/n)? n ##是否设置只查看用户密码，这里不设置
A view-only password is not used
New ‘CentOS77:1 (laopi)’ desktop is CentOS7:1
Creating default startup script /home/laopi/.vnc/xstartup
Creating default config /home/laopi/.vnc/config
Starting applications specified in /home/laopi/.vnc/xstartup
Log file is /home/laopi/.vnc/CentOS7:1.log
命令的执行是在需要远程登陆的laopi用户下执行的，同时也是第一次执行，会在此用户的主目录（/home/laopi/）下生成.vnc子目录和相应的配置文件，同时需要输入密码(vncviewer端用此用户访问的密码)，这个密码被加密保存在此用户主目录下的.vnc子目录（/home/laopi/.vnc/passwd）中；同时在用户主目录下的.vnc子目录中为用户自动建立xstartup配置文件（/home/laopi/.vnc/xstartup），在每次启动VNC服务时，都会读取该文件中的配置信息。上面的是添加普通用户通过VNC访问权限，如果想添加其他用户访问权限，需要在相应的用户命令提示符下进行上面的操作（如test用户也需要执行[test@ localhost ~]$ vncserver :1 在相应的主目录下生成对应的.vnc子目录）。对不同用户的访问密码更改也需要在相应的命令提示符下操作。
在/home/laopi/.vnc/目录下还有一个“CentOS7:1.pid”文件，这个文件记录着启动VNC后对应操作系统的进程号，用于停止VNC服务时准确定位进程号.
3、VNC服务使用的端口号与桌面号的关系
VNC服务使用的端口号与桌面号相关，VNC使用TCP端口从5900开始，对应关系如下
桌面号为“1” ---- 端口号为5901
桌面号为“2” ---- 端口号为5902
桌面号为“3” ---- 端口号为5903
…
基于上面的介绍，如果Linux开启了防火墙功能，就需要手工开启相应的端口.
防火墙设置，这里我就粗暴的把防火墙关了
systemctl stop firewalld.service ##停止firewall
systemctl disable firewalld.service ##禁止firewall开机启动
配置ssh密钥互认(oracle和grid两个用户分别以下操作)
chmod 700 ~/.ssh
1、node1
ssh-keygen -t rsa 一路按回车
ssh-keygen -t dsa 一路按回车
ssh-copy-id -i .ssh/id_dsa.pub node2
ssh-copy-id -i .ssh/id_rsa.pub node2
2、node2
ssh-keygen -t rsa 一路按回车
ssh-keygen -t dsa 一路按回车
ssh-copy-id -i .ssh/id_rsa.pub node1
ssh-copy-id -i .ssh/id_dsa.pub node1
cat .ssh/*.pub >> .ssh/authorized_keys
3、node1
cat .ssh/*.pub >> .ssh/authorized_keys
4、node1
ssh node1 date
ssh node1 date && ssh node1-priv date && ssh node2 date && ssh node2-priv date
5、node2
ssh node2 date
ssh node1 date && ssh node1-priv date && ssh node2 date && ssh node2-priv date
NTP配置
Node1&node2
Vi /etc/sysconfig/ntpd
Drop root to id ‘ntp:ntp’ by default.
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
systemctl restart ntpd
ntpq -p
remote refid st t when poll reach delay offset jitter
stratum2-1.ntp. 195.91.239.8 2 u 1 64 1 121.688 -0.965 6.374
de-user.deepini 195.13.23.5 3 u - 64 1 275.970 10.021 4.695
202.118.1.130 .PTP. 1 u 2 64 1 17.773 1.771 1.320
ntp1.ams1.nl.le 130.133.1.10 2 u - 64 1 335.141 -88.276 0.771
cvuqdisk-1.0.9-1.rpm报错
rpm -ivh cvuqdisk-1.0.9-1.rpm
4.2解压安装包
[grid@rac1 ~]$ unzip p13390677_112040_Linux-x86-64_3of7.zip
[grid@rac1 ~]$ ll
drwxr-xr-x 7 grid oinstall 156 8月 27 2013 grid
drwxr-xr-x 3 grid oinstall 18 7月 5 21:43 oradiag_grid
-rwxrwxr-x 1 grid oinstall 1205251894 2月 27 2015 p13390677_112040_Linux-x86-64_3of7.zip
[grid@rac1 ~]$ cd grid/
[grid@rac1 grid]$ ./runInstaller
两边执行自动修复脚本
依次再在两个节点用root用户执行两个脚本，先执行脚本1按节点一二顺序，再执行脚本2按节点一二顺序
在执行第二个脚本时会出现版本bug
要在新开窗口执行
dd if=/var/tmp/.oracle/npohasd of=/dev/null bs=1024 count=1
安装完成后显示的报错可以忽略
1.使用root用户登录系统，进入/tmp目录下,创建一个2G的swapfree文件
dd if=/dev/zero of=swapfree bs=32k count=65515
这个过程可能会比较久一点
2.将swapfree设置为Swap交互空间
mkswap swapfree
3.开启交换空间
swapon swapfree
4.查看Swap空间
free
5.设置交换分区开机启动
vim /etc/fstab
添加以下内容
/dec/hdb5 swap swap defaults 0 0
/tmp/swapfree swap swap defaults 0 0
rpm rpm -ivh oracleasm* --force --nodeps
4.3检查grid集群是否安装成功
[grid@rac1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[grid@rac1 ~]$ crsctl stat res -t
NAME TARGET STATE SERVER STATE_DETAILS
Local Resources
ora.CRS.dg
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.DATA.dg
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.LISTENER.lsnr
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.asm
ONLINE ONLINE rac1 Started
ONLINE ONLINE rac2 Started
ora.gsd
OFFLINE OFFLINE rac1
OFFLINE OFFLINE rac2
ora.net1.network
ONLINE ONLINE rac1
ONLINE ONLINE rac2
ora.ons
ONLINE ONLINE rac1
ONLINE ONLINE rac2
Cluster Resources
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac2
ora.cvu
1 ONLINE ONLINE rac1
ora.oc4j
1 ONLINE ONLINE rac2
ora.rac1.vip
1 ONLINE ONLINE rac1
ora.rac2.vip
1 ONLINE ONLINE rac2
ora.scan1.vip
1 ONLINE ONLINE rac2
ora.zzw.db
1 ONLINE ONLINE rac1 Open
2 ONLINE ONLINE rac2 Open
5.安装数据库软件
上传数据库压缩包到oracle家目录下并赋权
[oracle@rac1 ~]$ ll
-rwxrwxr-x 1 oracle oinstall 1212620273 Nov 7 2019 p13390677_112040_Linux-x86-64_1of7.zip
-rwxrwxr-x 1 oracle oinstall 113112960 Nov 7 2019 p13390677_112040_Linux-x86-64_2of7.zip
开始安装
[oracle@rac1 ~]$ cd database/
[oracle@rac1 database]$ ./runInstaller
执行以上脚本
6 安装asmca
使用grid用户执行asmca命令
[grid@rac1 ~]$ asmca
7 安装dbca
[oracle@rac1 ~]$ dbca
