gs_probackup对物理备份gs_basebackup的增强

1. 支持PTRACK增量备份
2. 支持增量恢复：基于页级增量恢复
3. Merge合并：合并增量备份
4. 数据备份有效性自动检测，无需实际的数据恢复
5. 数据库实例checksum检测(需要打开checksum）
6. 备份保留策略：按时间或数量保留WAL归档及备份
7. 多线程并行化：支持backup、restore、merge等命令
8. 支持远程操作：支持SSH方式远程备份或恢复实例
9. 支持从standby备份，避免主库额外负载压力
10. 数据存储在外部目录：与PGDATA保持独立
11. 备份元数据管理：命令配置持久化到文件

开启配置

使用PTRACK增量备份，需在postgresql.conf中手动添加参数“enable_cbm_tracking = on”。

gs_guc reload -D /opt/mogdb/data  -c "enable_cbm_tracking = on"

需要开启归档

初始化备份路径

gs_probackup init -B /data/backup_dir/

会在指定路径下生成backups和wal两个文件夹

初始化备份实例

在备份路径backup-path内初始化一个新的备份实例，并生成pg_probackup.conf配置文件，该文件保存了指定数据目录pgdata-path的gs_probackup设置。

gs_probackup add-instance -B /data/backup_dir -D /opt/mogdb/data --instance instance_local

在路径/data/backup_dir/backups/instance_local下生成pg_probackup.conf配置文件

全备

gs_probackup  backup -B /data/backup_dir --instance instance_local -d postgres -b FULL 

INFO: Backup start, gs_probackup version: 2.4.2, instance: instance_local, backup ID: SUFXAM, backup mode: FULL, wal mode: STREAM, remote: false, compress-algorithm: none, compress-level: 1
LOG: Backup destination is initialized
LOG: This openGauss instance was initialized with data block checksums. Data block corruption will be detected
LOG: Database backup start
LOG: started streaming WAL at 1/17000000 (timeline 1)
[2025-04-09 15:39:58]: check identify system success                                                
[2025-04-09 15:39:58]: send START_REPLICATION 1/17000000 success                                    
[2025-04-09 15:39:58]: keepalive message is received                                                
[2025-04-09 15:39:58]: keepalive message is received                                                
INFO: PGDATA size: 649MB
INFO: Start transferring data files
LOG: Creating page header map "/data/backup_dir/backups/instance_local/SUFXAM/page_header_map"
INFO: Data files are transferred, time elapsed: 3s
INFO: wait for pg_stop_backup()
INFO: pg_stop backup() successfully executed
LOG: stop_lsn: 1/170001E8
LOG: Looking for LSN 1/170001E8 in segment: 000000010000000100000017
LOG: Found WAL segment: /data/backup_dir/backups/instance_local/SUFXAM/database/pg_xlog/000000010000000100000017
LOG: Thread [0]: Opening WAL segment "/data/backup_dir/backups/instance_local/SUFXAM/database/pg_xlog/000000010000000100000017"
LOG: Found LSN: 1/170001E8
LOG: finished streaming WAL at 1/18000000 (timeline 1)
LOG: Getting the Recovery Time from WAL
LOG: Thread [0]: Opening WAL segment "/data/backup_dir/backups/instance_local/SUFXAM/database/pg_xlog/000000010000000100000017"
INFO: Syncing backup files to disk
INFO: Backup files are synced, time elapsed: 0
INFO: Validating backup SUFXAM
INFO: Backup SUFXAM data files are valid
INFO: Backup SUFXAM resident size: 665MB
INFO: Backup SUFXAM completed

查看备份情况

gs_probackup show -B /data/backup_dir/ --instance instance_local

增量备份

增加一些模拟数据

MogDB=# create table t1add(id int);
CREATE TABLE
MogDB=# insert into  t1add values(111111);
INSERT 0 1

进行增量备份

gs_probackup  backup -B /data/backup_dir/ --instance instance_local -d postgres -b PTRACK

INFO: Backup start, gs_probackup version: 2.4.2, instance: instance_local, backup ID: SUFXHO, backup mode: PTRACK, wal mode: STREAM, remote: false, compress-algorithm: none, compress-level: 1
LOG: Backup destination is initialized
LOG: This openGauss instance was initialized with data block checksums. Data block corruption will be detected
LOG: Database backup start
LOG: Latest valid FULL backup: SUFXAM
INFO: Parent backup: SUFXAM
LOG: started streaming WAL at 1/19000000 (timeline 1)
[2025-04-09 15:44:12]: check identify system success                                                
[2025-04-09 15:44:12]: send START_REPLICATION 1/19000000 success                                    
[2025-04-09 15:44:12]: keepalive message is received                                                
[2025-04-09 15:44:12]: keepalive message is received                                                
INFO: PGDATA size: 649MB
LOG: Current tli: 1
LOG: Parent start_lsn: 1/17000028
LOG: start_lsn: 1/19000028
INFO: Extracting pagemap of changed blocks
INFO: change bitmap start lsn location is 1/17000028
INFO: change bitmap end lsn location is 00000001/19000028
INFO: Pagemap successfully extracted, time elapsed: 0 sec
INFO: Start transferring data files
LOG: Creating page header map "/data/backup_dir/backups/instance_local/SUFXHO/page_header_map"
INFO: Data files are transferred, time elapsed: 2s
INFO: wait for pg_stop_backup()
INFO: pg_stop backup() successfully executed
LOG: stop_lsn: 1/190001E8
LOG: Looking for LSN 1/190001E8 in segment: 000000010000000100000019
LOG: Found WAL segment: /data/backup_dir/backups/instance_local/SUFXHO/database/pg_xlog/000000010000000100000019
LOG: Thread [0]: Opening WAL segment "/data/backup_dir/backups/instance_local/SUFXHO/database/pg_xlog/000000010000000100000019"
LOG: Found LSN: 1/190001E8
LOG: finished streaming WAL at 1/1A000000 (timeline 1)
LOG: Getting the Recovery Time from WAL
LOG: Thread [0]: Opening WAL segment "/data/backup_dir/backups/instance_local/SUFXHO/database/pg_xlog/000000010000000100000019"
INFO: Syncing backup files to disk
INFO: Backup files are synced, time elapsed: 0
INFO: Validating backup SUFXHO
INFO: Backup SUFXHO data files are valid
INFO: Backup SUFXHO resident size: 275MB
INFO: Backup SUFXHO completed

查看备份

 gs_probackup show -B /data/backup_dir/ --instance instance_local
=================================================================================================================================================
 Instance        Version  ID      Recovery Time           Mode    WAL Mode  TLI  Time   Data   WAL  Zratio  Start LSN   Stop LSN    Type  Status 
=================================================================================================================================================
 instance_local  9.2      SUFXHO  2025-04-09 15:44:14+08  PTRACK  STREAM    1/1    5s  259MB  16MB    1.00  1/19000028  1/190001E8  FILE  OK     
 instance_local  9.2      SUFXAM  2025-04-09 15:40:01+08  FULL    STREAM    1/0    5s  649MB  16MB    1.00  1/17000028  1/170001E8  FILE  OK  

合并备份

gs_probackup merge -B /data/backup_dir/ --instance instance_local  -i SUFXHO

将指定的增量备份与其父完全备份之间的所有增量备份合并到父完全备份。父完全备份将接收所有合并的数据，而已合并的增量备份将作为冗余被删除。

INFO: Merge started
INFO: Merging backup SUFXHO with parent chain
INFO: Validate parent chain for backup SUFXHO
INFO: Validating backup SUFXAM
INFO: Backup SUFXAM data files are valid
INFO: Validating backup SUFXHO
INFO: Backup SUFXHO data files are valid
LOG: Restore directories and symlinks... in /data/backup_dir/backups/instance_local/SUFXAM/database
INFO: Start merging backup files
LOG: Creating page header map "/data/backup_dir/backups/instance_local/SUFXAM/page_header_map_tmp"
INFO: Backup files are successfully merged, time elapsed: 7s
INFO: Delete: SUFXHO 2025-04-09 15:44:14+08
LOG: Rename /data/backup_dir/backups/instance_local/SUFXAM to /data/backup_dir/backups/instance_local/SUFXHO
INFO: Rename merged full backup SUFXAM to SUFXHO
INFO: Validating backup SUFXHO
INFO: Backup SUFXHO data files are valid
INFO: Merge of backup SUFXHO completed

再次查看备份，只有一个备份了

gs_probackup show -B /data/backup_dir/ --instance instance_local
================================================================================================================================================
 Instance        Version  ID      Recovery Time           Mode  WAL Mode  TLI  Time    Data   WAL  Zratio  Start LSN   Stop LSN    Type  Status 
================================================================================================================================================
 instance_local  9.2      SUFXHO  2025-04-09 15:44:14+08  FULL  STREAM    1/0    7s  1030MB  16MB    1.00  1/19000028  1/190001E8  FILE  OK  

恢复

模拟主库丢数据且宕机
(1)drop table t1add;
(2)使用restore子命令前，应先停止gaussdb进程
(3)
cd /opt/mogdb/
mv data data.bak
mkdir data

gs_probackup restore -B /data/backup_dir/ --instance instance_local  -D /opt/mogdb/data

LOG: Restore begin.
LOG: there is no file tablespace_map
LOG: check tablespace directories of backup SUFXHO
LOG: check external directories of backup SUFXHO
INFO: Validating backup SUFXHO
INFO: Backup SUFXHO data files are valid
LOG: Thread [1]: Opening WAL segment "/data/backup_dir/backups/instance_local/SUFXHO/database/pg_xlog/000000010000000100000019"
INFO: Backup SUFXHO WAL segments are valid
INFO: Backup SUFXHO is valid.
INFO: Restoring the database from backup at 2025-04-09 15:44:12+08
LOG: there is no file tablespace_map
LOG: Restore directories and symlinks... in /opt/mogdb/data
INFO: Start restoring backup files. PGDATA size: 665MB
LOG: Start thread 1
INFO: Backup files are restored. Transfered bytes: 1046MB, time elapsed: 3s
INFO: Restore incremental ratio (less is better): 157% (1046MB/665MB)
INFO: Syncing restored files to disk
INFO: Restored backup files are synced, time elapsed: 0
INFO: Restore of backup SUFXHO completed.

启动数据库

gs_om -t start

查看之前已经删掉的数据库已恢复。内容与增量备份后数据一致。

MogDB=# select * from t1add;
   id   
--------
 111111
(1 row)

清理备份

gs_probackup del-instance -B /data/backup_dir/ --instance instance_local
INFO: Delete: SUFXHO 2025-04-09 15:44:14+08
INFO: Instance 'instance_local' successfully deleted