greenplum整个集群是由多台服务器组合而成，任何一台服务都有可能发生软件或硬件故障，我们一起来模拟一下任何一个节点或服务器故障后，greenplumn的容错及恢复方法.
本文主要说明master或standby master故障后的处理办法。
master状态检测
使用gpstate -f
或查询select * from gp_segment_configuration order by 2,1;
即可。
Greenplum中恢复故障的Master命令gpactivatestandby
https://www.xmmup.com/greenplumzhonghuifuguzhangdemasterminglinggpactivatestandby.html
master恢复示例
环境说明：https://www.xmmup.com/mppjiagouzhigreenplumdeanzhuangpeizhigaojiban.html#huan_jing_shen_qing
正常集群状态
在master查看数据库当前的状态：
1[gpadmin@mdw1 ~]$ gpstate
220230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-Starting gpstate with args:
320230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source'
420230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 20 2022 08:02:23'
520230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-Obtaining Segment details from master...
620230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-Gathering data from segments...
720230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-Greenplum instance status summary
820230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-----------------------------------------------------
920230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Master instance = Active
1020230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Master standby = mdw2
1120230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Standby master state = Standby host passive
1220230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total segment instance count from metadata = 32
1320230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-----------------------------------------------------
1420230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Primary Segment Status
1520230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-----------------------------------------------------
1620230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total primary segments = 16
1720230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total primary segment valid (at master) = 16
1820230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total primary segment failures (at master) = 0
1920230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0
2020230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of postmaster.pid files found = 16
2120230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0
2220230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 16
2320230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0
2420230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of /tmp lock files found = 16
2520230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number postmaster processes missing = 0
2620230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number postmaster processes found = 16
2720230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-----------------------------------------------------
2820230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Mirror Segment Status
2920230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-----------------------------------------------------
3020230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total mirror segments = 16
3120230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total mirror segment valid (at master) = 16
3220230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total mirror segment failures (at master) = 0
3320230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0
3420230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of postmaster.pid files found = 16
3520230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0
3620230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 16
3720230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0
3820230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number of /tmp lock files found = 16
3920230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number postmaster processes missing = 0
4020230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number postmaster processes found = 16
4120230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number mirror segments acting as primary segments = 0
4220230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:- Total number mirror segments acting as mirror segments = 16
4320230131:16:37:05:021525 gpstate:mdw1:gpadmin-[INFO]:-----------------------------------------------------
44
故障情况1：master 服务器故障
当master节点故障后，我们需要激活standby节点作为新的master节点（如果服务器配置有VIP，那么把vip也切换到standby服务器）
在激活standby节点的可以直接指定新的standby节点，也可以等原master服务器恢复后，指定原master节点为standby节点
关闭master节点
1[root@lhrxxt ~]# docker stop mdw1
2mdw1
激活standby节点
1[root@mdw2 /]# su - gpadmin
2Last login: Tue Jan 31 16:05:54 CST 2023 on pts/1
3[gpadmin@mdw2 ~]$
4[gpadmin@mdw2 ~]$ gpactivatestandby -d /opt/greenplum/data/master/gpseg-1
520230201:13:47:23:016042 gpactivatestandby:mdw2:gpadmin-[CRITICAL]:-PGPORT environment variable not set.
6[gpadmin@mdw2 ~]$ export PGPORT=5432
7[gpadmin@mdw2 ~]$ gpactivatestandby -d /opt/greenplum/data/master/gpseg-1
820230201:13:47:43:016069 gpactivatestandby:mdw2:gpadmin-[DEBUG]:-Running Command: echo 'START_CMD_OUTPUT';ps -ef | grep postgres | grep -v grep | awk '{print $2}' | grep \`cat /opt/greenplum/data/master/gpseg-1/postmaster.pid | head -1\` || echo -1
920230201:13:47:43:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
1020230201:13:47:43:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Standby data directory = /opt/greenplum/data/master/gpseg-1
1120230201:13:47:43:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Standby port = 5432
1220230201:13:47:43:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Standby running = yes
1320230201:13:47:43:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Force standby activation = no
1420230201:13:47:43:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
1520230201:13:47:43:016069 gpactivatestandby:mdw2:gpadmin-[DEBUG]:-Running Command: cat /tmp/.s.PGSQL.5432.lock
16Do you want to continue with standby master activation? Yy|Nn (default=N):
17> y
1820230201:13:47:51:016069 gpactivatestandby:mdw2:gpadmin-[DEBUG]:-Running Command: ps -ef | grep postgres | grep -v grep | awk '{print $2}' | grep `cat /opt/greenplum/data/master/gpseg-1/postmaster.pid | head -1` || echo -1
1920230201:13:47:51:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-found standby postmaster process
2020230201:13:47:51:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Promoting standby...
2120230201:13:47:51:016069 gpactivatestandby:mdw2:gpadmin-[DEBUG]:-Running Command: pg_ctl promote -D /opt/greenplum/data/master/gpseg-1
2220230201:13:47:51:016069 gpactivatestandby:mdw2:gpadmin-[DEBUG]:-Waiting for connection...
2320230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Standby master is promoted
2420230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Reading current configuration...
2520230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[DEBUG]:-Connecting to dbname='postgres'
2620230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
2720230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-The activation of the standby master has completed successfully.
2820230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-mdw2 is now the new primary master.
2920230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-You will need to update your user access mechanism to reflect
3020230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-the change of master hostname.
3120230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Do not re-start the failed master while the fail-over master is
3220230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-operational, this could result in database corruption!
3320230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-MASTER_DATA_DIRECTORY is now /opt/greenplum/data/master/gpseg-1 if
3420230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-this has changed as a result of the standby master activation, remember
3520230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-to change this in any startup scripts etc, that may be configured
3620230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-to set this value.
3720230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-MASTER_PORT is now 5432, if this has changed, you
3820230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-may need to make additional configuration changes to allow access
3920230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-to the Greenplum instance.
4020230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Refer to the Administrator Guide for instructions on how to re-activate
4120230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-the master to its previous state once it becomes available.
4220230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-Query planner statistics must be updated on all databases
4320230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-following standby master activation.
4420230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:-When convenient, run ANALYZE against all user databases.
4520230201:13:47:52:016069 gpactivatestandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
46[gpadmin@mdw2 ~]$
standby master启用VIP:
1[root@mdw2 ~]$ ip addr add 172.72.6.56/16 dev eth0 label eth0:1
2[root@mdw2 /]# ifconfig
3eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
4 inet 172.72.6.51 netmask 255.255.0.0 broadcast 172.72.255.255
5 ether 02:42:ac:48:06:33 txqueuelen 0 (Ethernet)
6 RX packets 94731 bytes 7030183 (6.7 MiB)
7 RX errors 0 dropped 0 overruns 0 frame 0
8 TX packets 37837 bytes 166369481 (158.6 MiB)
9 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
10
11eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
12 inet 172.72.6.56 netmask 255.255.0.0 broadcast 0.0.0.0
13 ether 02:42:ac:48:06:33 txqueuelen 0 (Ethernet)
14
15lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
16 inet 127.0.0.1 netmask 255.0.0.0
17 loop txqueuelen 1000 (Local Loopback)
18 RX packets 1682 bytes 463712 (452.8 KiB)
19 RX errors 0 dropped 0 overruns 0 frame 0
20 TX packets 1682 bytes 463712 (452.8 KiB)
21 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
22
23[root@mdw2 /]#
24
25
26[root@lhrxxt ~]# psql -U gpadmin -h 172.72.6.56 -d postgres
27Password for user gpadmin:
28psql (13.9, server 9.4.26)
29Type "help" for help.
30
31postgres=# \l
32 List of databases
33 Name | Owner | Encoding | Collate | Ctype | Access privileges
34-----------+---------+----------+------------+------------+---------------------
35 lhrgpdb | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 |
36 postgres | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 |
37 template0 | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/gpadmin +
38 | | | | | gpadmin=CTc/gpadmin
39 template1 | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 | =c/gpadmin +
40 | | | | | gpadmin=CTc/gpadmin
41 testdb | gpadmin | UTF8 | en_US.utf8 | en_US.utf8 |
42(5 rows)
43
44postgres=#
45
此时，集群即可正常使用。
修复mdw1节点，然后指定为新的standby节点
我们指定原master节点为新的standby节点服务器
需要先删除原master的数据文件，然后重新执行初始化standby节点即可
错误操作：在mdw1节点启动整个GreenPlum集群会报错，修复集群也会报错：
1[gpadmin@mdw1 ~]$ gpstate
220230201:13:56:07:000419 gpstate:mdw1:gpadmin-[INFO]:-Starting gpstate with args:
320230201:13:56:07:000419 gpstate:mdw1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source'
420230201:13:56:07:000419 gpstate:mdw1:gpadmin-[CRITICAL]:-gpstate failed. (Reason='could not connect to server: Connection refused
5 Is the server running on host "localhost" (127.0.0.1) and accepting
6 TCP/IP connections on port 5432?
7could not connect to server: Cannot assign requested address
8 Is the server running on host "localhost" (::1) and accepting
9 TCP/IP connections on port 5432?
10') exiting...
11[gpadmin@mdw1 ~]$ gpstart
1220230201:13:56:34:000462 gpstart:mdw1:gpadmin-[INFO]:-Starting gpstart with args:
1320230201:13:56:34:000462 gpstart:mdw1:gpadmin-[INFO]:-Gathering information and validating the environment...
1420230201:13:56:34:000462 gpstart:mdw1:gpadmin-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source'
1520230201:13:56:34:000462 gpstart:mdw1:gpadmin-[INFO]:-Greenplum Catalog Version: '301908232'
1620230201:13:56:34:000462 gpstart:mdw1:gpadmin-[INFO]:-Starting Master instance in admin mode
1720230201:13:56:34:000462 gpstart:mdw1:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
1820230201:13:56:34:000462 gpstart:mdw1:gpadmin-[INFO]:-Obtaining Segment details from master...
1920230201:13:56:34:000462 gpstart:mdw1:gpadmin-[INFO]:-Setting new master era
2020230201:13:56:34:000462 gpstart:mdw1:gpadmin-[INFO]:-Master Started...
2120230201:13:56:35:000462 gpstart:mdw1:gpadmin-[INFO]:-Master Stopped...
2220230201:13:56:35:000462 gpstart:mdw1:gpadmin-[ERROR]:-gpstart error: Standby activated, this node no more can act as master.
23[gpadmin@mdw1 ~]$ gpinitstandby -a -s mdw1
2420230201:13:57:06:000532 gpinitstandby:mdw1:gpadmin-[ERROR]:-Failed to retrieve configuration information from the master.
2520230201:13:57:06:000532 gpinitstandby:mdw1:gpadmin-[ERROR]:-Failed to create standby
2620230201:13:57:06:000532 gpinitstandby:mdw1:gpadmin-[ERROR]:-Error initializing standby master: could not connect to server: Connection refused
27 Is the server running on host "localhost" (127.0.0.1) and accepting
28 TCP/IP connections on port 5432?
29could not connect to server: Cannot assign requested address
30 Is the server running on host "localhost" (::1) and accepting
31 TCP/IP connections on port 5432?
32
我们应该在md2上操作：
1[gpadmin@mdw2 ~]$ gpinitstandby -a -s mdw1
220230201:13:57:16:016792 gpinitstandby:mdw2:gpadmin-[INFO]:-Validating environment and parameters for standby initialization...
320230201:13:57:16:016792 gpinitstandby:mdw2:gpadmin-[INFO]:-Checking for data directory /opt/greenplum/data/master/gpseg-1 on mdw1
420230201:13:57:16:016792 gpinitstandby:mdw2:gpadmin-[ERROR]:-Data directory already exists on host mdw1
520230201:13:57:16:016792 gpinitstandby:mdw2:gpadmin-[ERROR]:-Failed to create standby
620230201:13:57:16:016792 gpinitstandby:mdw2:gpadmin-[ERROR]:-Error initializing standby master: master data directory exists
7
8
9-- 注意：会报错，需要去mdw1上删除原来的数据目录：
10[gpadmin@mdw1 master]$ cd /opt/greenplum/data/master/
11[gpadmin@mdw1 master]$ rm -rf gpseg-1/
12[gpadmin@mdw1 master]$ ll
13
14
15
16-- 继续在mdw2上进行修复：
17[gpadmin@mdw2 ~]$ gpinitstandby -a -s mdw1
1820230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Validating environment and parameters for standby initialization...
1920230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Checking for data directory /opt/greenplum/data/master/gpseg-1 on mdw1
2020230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
2120230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master initialization parameters
2220230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
2320230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum master hostname = mdw2
2420230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum master data directory = /opt/greenplum/data/master/gpseg-1
2520230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum master port = 5432
2620230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master hostname = mdw1
2720230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master port = 5432
2820230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master data directory = /opt/greenplum/data/master/gpseg-1
2920230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum update system catalog = On
3020230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Syncing Greenplum Database extensions to standby
3120230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-The packages on mdw1 are consistent.
3220230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Adding standby master to catalog...
3320230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Database catalog updated successfully.
3420230201:13:58:30:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Updating pg_hba.conf file...
3520230201:13:58:31:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-pg_hba.conf files updated successfully.
3620230201:13:58:38:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Starting standby master
3720230201:13:58:38:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Checking if standby master is running on host: mdw1 in directory: /opt/greenplum/data/master/gpseg-1
3820230201:13:58:39:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Cleaning up pg_hba.conf backup files...
3920230201:13:58:39:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Backup files of pg_hba.conf cleaned up successfully.
4020230201:13:58:39:016867 gpinitstandby:mdw2:gpadmin-[INFO]:-Successfully created standby master on mdw1
修复完成后的状态：
1[gpadmin@mdw2 ~]$ gpconfig -s max_connections
2Values on all segments are consistent
3GUC : max_connections
4Master value: 500
5Segment value: 1500
6[gpadmin@mdw2 ~]$ gpstate
720230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-Starting gpstate with args:
820230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source'
920230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 20 2022 08:02:23'
1020230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-Obtaining Segment details from master...
1120230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-Gathering data from segments...
1220230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-Greenplum instance status summary
1320230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-----------------------------------------------------
1420230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Master instance = Active
1520230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Master standby = mdw1
1620230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Standby master state = Standby host passive
1720230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total segment instance count from metadata = 32
1820230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-----------------------------------------------------
1920230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Primary Segment Status
2020230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-----------------------------------------------------
2120230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total primary segments = 16
2220230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total primary segment valid (at master) = 16
2320230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total primary segment failures (at master) = 0
2420230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0
2520230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of postmaster.pid files found = 16
2620230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0
2720230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 16
2820230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0
2920230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of /tmp lock files found = 16
3020230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number postmaster processes missing = 0
3120230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number postmaster processes found = 16
3220230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-----------------------------------------------------
3320230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Mirror Segment Status
3420230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-----------------------------------------------------
3520230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total mirror segments = 16
3620230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total mirror segment valid (at master) = 16
3720230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total mirror segment failures (at master) = 0
3820230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0
3920230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of postmaster.pid files found = 16
4020230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0
4120230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 16
4220230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0
4320230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number of /tmp lock files found = 16
4420230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number postmaster processes missing = 0
4520230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number postmaster processes found = 16
4620230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number mirror segments acting as primary segments = 0
4720230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:- Total number mirror segments acting as mirror segments = 16
4820230201:13:59:04:016970 gpstate:mdw2:gpadmin-[INFO]:-----------------------------------------------------
49[gpadmin@mdw2 ~]$ gpstate -f
5020230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:-Starting gpstate with args: -f
5120230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source'
5220230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 20 2022 08:02:23'
5320230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:-Obtaining Segment details from master...
5420230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:-Standby master details
5520230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:-----------------------
5620230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:- Standby address = mdw1
5720230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:- Standby data directory = /opt/greenplum/data/master/gpseg-1
5820230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:- Standby port = 5432
5920230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:- Standby PID = 1061
6020230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:- Standby status = Standby host passive
6120230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
6220230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:--pg_stat_replication
6320230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
6420230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:--WAL Sender State: streaming
6520230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:--Sync state: sync
6620230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:--Sent Location: 0/2C000000
6720230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:--Flush Location: 0/2C000000
6820230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:--Replay Location: 0/2C000000
6920230201:13:59:15:017051 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
70
故障情况2：standby 服务器故障
情况1：能正常启动
1[gpadmin@mdw2 ~]$ gpstate -f
220230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:-Starting gpstate with args: -f
320230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source'
420230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 20 2022 08:02:23'
520230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:-Obtaining Segment details from master...
620230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:-Standby master details
720230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:-----------------------
820230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:- Standby address = mdw1
920230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:- Standby data directory = /opt/greenplum/data/master/gpseg-1
1020230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:- Standby port = 5432
1120230201:15:08:57:020835 gpstate:mdw2:gpadmin-[WARNING]:-Standby PID = 1061 No socket connection or lock file (/tmp/.s.PGSQL.5432.lock) found for port 5432 <<<<<<<<
1220230201:15:08:57:020835 gpstate:mdw2:gpadmin-[WARNING]:-Standby status = Status could not be determined <<<<<<<<
1320230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
1420230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:--pg_stat_replication
1520230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
1620230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:-No entries found.
1720230201:15:08:57:020835 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
18
19
20[gpadmin@mdw1 ~]$ nohup /usr/local/greenplum-db-6.23.0/bin/postgres -D /opt/greenplum/data/master/gpseg-1 -p 5432 -E &
21[1] 2385
22
23[gpadmin@mdw2 ~]$ gpstate -f
2420230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:-Starting gpstate with args: -f
2520230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source'
2620230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 20 2022 08:02:23'
2720230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:-Obtaining Segment details from master...
2820230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:-Standby master details
2920230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:-----------------------
3020230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:- Standby address = mdw1
3120230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:- Standby data directory = /opt/greenplum/data/master/gpseg-1
3220230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:- Standby port = 5432
3320230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:- Standby PID = 2385
3420230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:- Standby status = Standby host passive
3520230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
3620230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:--pg_stat_replication
3720230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
3820230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:--WAL Sender State: streaming
3920230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:--Sync state: sync
4020230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:--Sent Location: 0/2C0000A8
4120230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:--Flush Location: 0/2C0000A8
4220230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:--Replay Location: 0/2C0000A8
4320230201:15:09:38:020946 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
44[gpadmin@mdw2 ~]$
45
情况2：数据库文件损坏不能正常启动
若数据库文件损坏不能正常启动，则需要将standby节点数据删除，然后重新初始化一下standby服务器即可
1[gpadmin@mdw1 master]$ ps -ef|grep green
2gpadmin 2385 2338 0 15:09 pts/1 00:00:00 /usr/local/greenplum-db-6.23.0/bin/postgres -D /opt/greenplum/data/master/gpseg-1 -p 5432 -E
3gpadmin 2740 2338 0 15:13 pts/1 00:00:00 grep --color=auto green
4[gpadmin@mdw1 master]$ kill -9 2385
5[gpadmin@mdw1 master]$
6[gpadmin@mdw1 ~]$ rm -rf /opt/greenplum/data/master/gpseg-1
7[gpadmin@mdw1 ~]$ cd /opt/greenplum/data/master/
8[gpadmin@mdw1 master]$ ll
9total 0
10[gpadmin@mdw2 ~]$ gpstate -f
1120230201:15:13:58:021233 gpstate:mdw2:gpadmin-[INFO]:-Starting gpstate with args: -f
1220230201:15:13:58:021233 gpstate:mdw2:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source'
1320230201:15:13:58:021233 gpstate:mdw2:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 20 2022 08:02:23'
1420230201:15:13:58:021233 gpstate:mdw2:gpadmin-[INFO]:-Obtaining Segment details from master...
1520230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:-Standby master details
1620230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:-----------------------
1720230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:- Standby address = mdw1
1820230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:- Standby data directory = /opt/greenplum/data/master/gpseg-1
1920230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:- Standby port = 5432
2020230201:15:13:59:021233 gpstate:mdw2:gpadmin-[WARNING]:-Standby PID = 0 <<<<<<<<
2120230201:15:13:59:021233 gpstate:mdw2:gpadmin-[WARNING]:-Standby status = Standby process not running <<<<<<<<
2220230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
2320230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:--pg_stat_replication
2420230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
2520230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:-No entries found.
2620230201:15:13:59:021233 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
27[gpadmin@mdw2 ~]$
修复：需要先删除再添加：
1[gpadmin@mdw2 ~]$ gpinitstandby -s mdw1 -S /opt/greenplum/data/master/gpseg-1 -P 5432
220230201:15:14:41:021292 gpinitstandby:mdw2:gpadmin-[INFO]:-Validating environment and parameters for standby initialization...
320230201:15:14:41:021292 gpinitstandby:mdw2:gpadmin-[ERROR]:-Standby master already configured
420230201:15:14:41:021292 gpinitstandby:mdw2:gpadmin-[INFO]:-If you want to start the stopped standby master, use the -n option
520230201:15:14:41:021292 gpinitstandby:mdw2:gpadmin-[ERROR]:-Failed to create standby
620230201:15:14:41:021292 gpinitstandby:mdw2:gpadmin-[ERROR]:-Error initializing standby master: standby master already configured
7[gpadmin@mdw2 ~]$ gpinitstandby -r
820230201:15:15:29:021341 gpinitstandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
920230201:15:15:29:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Warm master standby removal parameters
1020230201:15:15:29:021341 gpinitstandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
1120230201:15:15:29:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum master hostname = mdw2
1220230201:15:15:29:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum master data directory = /opt/greenplum/data/master/gpseg-1
1320230201:15:15:29:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum master port = 5432
1420230201:15:15:29:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master hostname = mdw1
1520230201:15:15:29:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master port = 5432
1620230201:15:15:29:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master data directory = /opt/greenplum/data/master/gpseg-1
17Do you want to continue with deleting the standby master? Yy|Nn (default=N):
18> y
1920230201:15:15:32:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Removing standby master from catalog...
2020230201:15:15:32:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Database catalog updated successfully.
2120230201:15:15:32:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Removing data directory on standby master...
2220230201:15:15:33:021341 gpinitstandby:mdw2:gpadmin-[INFO]:-Successfully removed standby master
23[gpadmin@mdw2 ~]$ gpinitstandby -s mdw1 -S /opt/greenplum/data/master/gpseg-1 -P 5432
2420230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Validating environment and parameters for standby initialization...
2520230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Checking for data directory /opt/greenplum/data/master/gpseg-1 on mdw1
2620230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
2720230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master initialization parameters
2820230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:------------------------------------------------------
2920230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum master hostname = mdw2
3020230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum master data directory = /opt/greenplum/data/master/gpseg-1
3120230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum master port = 5432
3220230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master hostname = mdw1
3320230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master port = 5432
3420230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum standby master data directory = /opt/greenplum/data/master/gpseg-1
3520230201:15:15:37:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Greenplum update system catalog = On
36Do you want to continue with standby master initialization? Yy|Nn (default=N):
37> y
3820230201:15:15:42:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Syncing Greenplum Database extensions to standby
3920230201:15:15:42:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-The packages on mdw1 are consistent.
4020230201:15:15:42:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Adding standby master to catalog...
4120230201:15:15:42:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Database catalog updated successfully.
4220230201:15:15:42:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Updating pg_hba.conf file...
4320230201:15:15:43:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-pg_hba.conf files updated successfully.
4420230201:15:15:50:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Starting standby master
4520230201:15:15:50:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Checking if standby master is running on host: mdw1 in directory: /opt/greenplum/data/master/gpseg-1
4620230201:15:15:51:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Cleaning up pg_hba.conf backup files...
4720230201:15:15:51:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Backup files of pg_hba.conf cleaned up successfully.
4820230201:15:15:51:021371 gpinitstandby:mdw2:gpadmin-[INFO]:-Successfully created standby master on mdw1
49[gpadmin@mdw2 ~]$ gpstate -f
5020230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:-Starting gpstate with args: -f
5120230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source'
5220230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 9.4.26 (Greenplum Database 6.23.0 build commit:5b5e432f35f92a40c18dffe4e5bca94790aae83c Open Source) on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Dec 20 2022 08:02:23'
5320230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:-Obtaining Segment details from master...
5420230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:-Standby master details
5520230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:-----------------------
5620230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:- Standby address = mdw1
5720230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:- Standby data directory = /opt/greenplum/data/master/gpseg-1
5820230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:- Standby port = 5432
5920230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:- Standby PID = 3328
6020230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:- Standby status = Standby host passive
6120230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
6220230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:--pg_stat_replication
6320230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
6420230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:--WAL Sender State: streaming
6520230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:--Sync state: sync
6620230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:--Sent Location: 0/34000000
6720230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:--Flush Location: 0/34000000
6820230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:--Replay Location: 0/34000000
6920230201:15:16:02:021460 gpstate:mdw2:gpadmin-[INFO]:--------------------------------------------------------------
70[gpadmin@mdw2 ~]$
总结
1、若master故障，则直接在standby master上执行如下命令将standby master激活为主master：
1gpactivatestandby -d /opt/greenplum/data/master/gpseg-1
若有VIP也需要切换VIP。
然后原来的master节点可以删除数据后，然后作为新的standby加入：
1gpinitstandby -a -s mdw1
2、若standby master故障，则分情况：若是宕机，其实可以正常启动，使用如下命令正常启动：
1nohup /usr/local/greenplum-db-6.23.0/bin/postgres -D /opt/greenplum/data/master/gpseg-1 -p 5432 -E &
若数据库文件损坏不能正常启动，则需要将standby节点数据删除，然后重新初始化一下standby服务器即可，需要先删除再添加：
1gpinitstandby -r
2gpinitstandby -s mdw1 -S /opt/greenplum/data/master/gpseg-1 -P 5432
3、master和standby master就是PG中的主从复制，要想实现自动故障转移，则需要借助keepalived或repmgr或patroni或pgpool实现高可用自动切换。
参考
https://blog.csdn.net/weixin_33724570/article/details/89720415
https://www.xmmup.com/greenplumshujukugaokeyongxinggaishu.html#Master_jing_xiang
https://zhuanlan.zhihu.com/p/165983070
文章转载自DB宝，如果涉嫌侵权，请发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。
