作者:邓琼
就职于中电福富信息科技有限公司,中国首批"PostgreSQL ACE伙伴"。主导并参与某省电信公司Oracle到PG的迁移工作,长期致力于电信行业的PostgreSQL技术推广工作。
一、ora2pg迁移说明
官方ora2pg支持的提供的支持类型有以下类:
TYPE
TABLE,VIEW,GRANT,SEQUENCE,TABLESPACE,PROCEDURE,TRIGGER,FUNCTION,PACKAGE,PARTITION,TYPE,MVIEW,QUERY,DBLINK,SYNONYM,DIRECTORY,TEST,TEST_VIEW
实际在导出function和package会报错,不建议导出function和package,具体报错:
function失败
[root@oracle ora2pg]# ora2pg -c dqiong_function.confe option must be (TABLE, VIEW, GRANT, SEQUENCE, TRIGGER, PACKAGE, FUNCTION,PROCEDURE, PARTITION, TYPE, INSERT, COPY, TABLESPACE, SHOW_REPORT, SHOW_VERSION,SHOW_SCHEMA, SHOW_TABLE, SHOW_COLUMN, SHOW_ENCODING, FDW, MVIEW, QUERY, KETTLE,DBLINK, SYNONYM, DIRECTORY, LOAD, TEST, TEST_VIEW), unknown function
package失败
[root@oracle ora2pg]# ora2pg -c dqiong_package.confe option must be (TABLE, VIEW, GRANT, SEQUENCE, TRIGGER, PACKAGE, FUNCTION,PROCEDURE, PARTITION, TYPE, INSERT, COPY, TABLESPACE, SHOW_REPORT, SHOW_VERSION,SHOW_SCHEMA, SHOW_TABLE, SHOW_COLUMN, SHOW_ENCODING, FDW, MVIEW, QUERY, KETTLE,DBLINK, SYNONYM, DIRECTORY, LOAD, TEST, TEST_VIEW), unknown package
oracle对象结构导出的建议类型:
table
view
mview
procedure
sequence
trigger
type
ora2pg教程参考地址:http://ora2pg.darold.net/documentation.html
二、安装ora2pg
2.1 下载安装包
下载地址:
链接:https://pan.baidu.com/s/1ZreKcz5qhWhl7LVPlb1gdg
提取码:fchx
[root@oracle ora2gp]#-rwxrwxr-x 1 oracle oinstall 448855 Jun 9 12:18 DBD-Oracle-1.74.tar.gz-rwxrwxr-x 1 oracle oinstall 276607 Jun 9 12:18 DBD-Pg-3.12.3.tar.gz-rwxrwxr-x 1 oracle oinstall 612372 Jun 9 12:18 DBI-1.643.tar.gz-rwxrwxr-x 1 oracle oinstall 319638 Jun 9 12:18 ora2pg-20.0.tar.bz
2.2 安装OS依赖软件包
[root@oracle ~]# yum install perl-DBD-Pg perl perl-devel perl-DBI perl-CPANbzip2 perl-ExtUtils-eBuilder perl-ExtUtils-MakeMaker
2.3 安装ORACLE瘦客户端
2.3.1方法一
从Oracle官方网站下载oracle 12c的瘦客户端,然后进行安装并配置。
[root@oracle u01]# la u01/software/oracle/total 55Mdrwxr-xr-x 2 root root 4.0K Dec 3 15:04 .drwxr-xr-x 7 root root 76 Dec 3 15:04 ..-rw-r--r—1 root root 51M Nov 28 14:34 oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm-rw-r--r--1 root root 593K Nov 28 14:32 oracle-instantclient12.2-devel-12.2.0.1.0-1.x86_64.rpm-rw-r--r--1 root root 1.5M Nov 28 14:32 oracle-instantclient12.2-jdbc-12.2.0.1.0-1.x86_64.rpm-rw-r--r--1 root root 692K Nov 28 14:32 oracle-instantclient12.2-sqlplus-12.2.0.1.0-1.x86_64.rpm-rw-r--r--1 root root 922K Nov 28 14:35 oracle-instantclient12.2-tools-12.2.0.1.0-1.x86_64.rpm[root@oracle ~]# yum -y localinstall u01/software/oracle/oracle-instantclient12.2-*.rpm[root@oracle ~]# mkdir -p usr/lib/oracle/12.2/client64/network/admin
编辑root环境变量,加入以下内容:
[root@oracle ~]# vim ~/.bash_profileexport LD_LIBRARY_PATH=/usr/lib/oracle/12.2/client64/libexport NLS_LANG=AMERICAN_AMERICA.ZHS16GBKexport TNS_ADMIN=/usr/lib/oracle/12.2/client64/network/adminexport ORACLE_HOME=/usr/lib/oracle/12.2/client64/export PATH=$PATH:$ORACLE_HOME/bin:$HOME/bin[root@oracle ~]# source .bash_profile
2.3.2方法二
直接在oracle服务器端安装ora2pg工具,直接调用oracle服务端包即可。并用root用户配置环境变量:
export LD_LIBRARY_PATH=/u01/app/oracle/product/12.1.0/db_1/libexport NLS_LANG=AMERICAN_AMERICA.ZHS16GBKexport TNS_ADMIN=/u01/app/oracle/product/12.1.0/db_1/network/adminexport ORACLE_HOME=//u01/app/oracle/product/12.1.0/db_1export PATH=$PATH:$ORACLE_HOME/bin:$HOME/bin
2.4 安装ora2pg
[root@oracle ~]# tar -xjf ora2pg-20.0.tar.bz2[root@oracle ~]# cd ora2pg-20.0[root@oracle ora2pg-20.0]# perl Makefile.PLChecking if your kit is complete...Looks goodWriting Makefile for Ora2PgDone...------------------------------------------------------------------------------Please read documentation at http://ora2pg.darold.net/ before asking for help------------------------------------------------------------------------------Now type: make && make install[root@oracle ora2pg-20.0]# make && make installcp lib/Ora2Pg.pm blib/lib/Ora2Pg.pmcp lib/Ora2Pg/GEOM.pm blib/lib/Ora2Pg/GEOM.pmcp lib/Ora2Pg/PLSQL.pm blib/lib/Ora2Pg/PLSQL.pmcp lib/Ora2Pg/MySQL.pm blib/lib/Ora2Pg/MySQL.pmcp scripts/ora2pg blib/script/ora2pg/usr/bin/perl -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/ora2pgcp scripts/ora2pg_scanner blib/script/ora2pg_scanner/usr/bin/perl -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/ora2pg_scannerManifying blib/man3/ora2pg.3Installing usr/local/share/perl5/Ora2Pg.pmInstalling usr/local/share/perl5/Ora2Pg/GEOM.pmInstalling usr/local/share/perl5/Ora2Pg/PLSQL.pmInstalling usr/local/share/perl5/Ora2Pg/MySQL.pmInstalling usr/local/share/man/man3/ora2pg.3Installing usr/local/bin/ora2pgInstalling usr/local/bin/ora2pg_scannerInstalling default configuration file (ora2pg.conf.dist) to etc/ora2pgAppending installation info to usr/lib64/perl5/perllocal.pod
2.5 安装DBD::Oracle
[root@oracle ~] tar -xvzf DBD-Oracle-1.74.tar.gz[root@oracle~]cd home/ora2gp/DBD-Oracle-1.74[root@oracle DBD-Oracle-1.74]# perl Makefile.PL[root@oracle DBD-Oracle-1.74]# make && make install
2.6 创建ora2pg配置文件
2.6.1 创建数据库全部结构配置文件
支持的type类型:
-rw-r--r-- 1 root root 322 Jul 26 00:35 ora2pg_data.conf-rw-r--r-- 1 root root 2312 Jul 25 22:11 ora2pg_mview.conf-rw-r--r-- 1 root root 2320 Jul 25 21:57 ora2pg_procedure.conf-rw-r--r-- 1 root root 2319 Jul 25 21:49 ora2pg_sequence.conf-rw-r--r-- 1 root root 2313 Jul 25 21:48 ora2pg_table.conf-rw-r--r-- 1 root root 2316 Jul 25 22:15 ora2pg_trigger.conf-rw-r--r-- 1 root root 2311 Jul 25 21:48 ora2pg_type.conf-rw-r--r-- 1 root root 2311 Jul 25 21:47 ora2pg_view.conf
具体的配置如下:
#cat etc/ora2pg/ora2pg.conf.dist | grep -v ^# |grep -v ^$ >ora2pg_table.conf[root@test01 ora2pg]# vim ora2pg_table.confORACLE_HOME /u01/app/oracle/product/12.1.0/db_1ORACLE_DSN dbi:Oracle:host=192.168.0.9;sid=orcl;port=1521ORACLE_USER systemORACLE_PWD systemUSER_GRANTS 1DEBUG 0ORA_INITIAL_COMMAND commitEXPORT_SCHEMA 1SCHEMA demoCREATE_SCHEMA 0COMPILE_SCHEMA 0TYPE TABLEDISABLE_COMMENT 0EXTERNAL_TO_FDW 1TRUNCATE_TABLE 0USE_TABLESPACE 0REORDERING_COLUMNS 0CONTEXT_AS_TRGM 0FTS_INDEX_ONLY 1USE_UNACCENT 0USE_LOWER_UNACCENT 0DATADIFF 0DATADIFF_UPDATE_BY_PKEY 0DATADIFF_DEL_SUFFIX _delDATADIFF_UPD_SUFFIX _updDATADIFF_INS_SUFFIX _insDATADIFF_WORK_MEM 256 MBDATADIFF_TEMP_BUFFERS 512 MBKEEP_PKEY_NAMES 0PKEY_IN_CREATE 0FKEY_ADD_UPDATE neverFKEY_DEFERRABLE 0DEFER_FKEY 0DROP_FKEY 0DISABLE_SEQUENCE 0DISABLE_TRIGGERS 1PRESERVE_CASE 0INDEXES_RENAMING 0USE_INDEX_OPCLASS 0PREFIX_PARTITION 0PREFIX_SUB_PARTITION 1DISABLE_PARTITION 0WITH_OID 0ORA_RESERVED_WORDS audit,comment,referencesUSE_RESERVED_WORDS 0DISABLE_UNLOGGED 0PG_DSN dbi:Pg:dbname=demo;host=192.168.0.11;port=5432PG_USER demoPG_PWD demo123.comOUTPUT demo_table.sqlOUTPUT_DIR /home/ora2gpBZIP2FILE_PER_CONSTRAINT 0FILE_PER_INDEX 0FILE_PER_FKEYS 0FILE_PER_TABLE 0FILE_PER_FUNCTION 0STOP_ON_ERROR 1COPY_FREEZE 0CREATE_OR_REPLACE 1PG_NUMERIC_TYPE 1PG_INTEGER_TYPE 1DEFAULT_NUMERIC bigintENABLE_MICROSECOND 1TO_NUMBER_CONVERSION numericGEN_USER_PWD 0FORCE_OWNER 0FORCE_SECURITY_INVOKER 0DATA_LIMIT 10000NOESCAPE 0TRANSACTION serializableSTANDARD_CONFORMING_STRINGS 1NO_LOB_LOCATOR 1XML_PRETTY 0LOG_ON_ERROR 0TRIM_TYPE BOTHINTERNAL_DATE_MAX 49FUNCTION_CHECK 1NO_BLOB_EXPORT 0DATA_EXPORT_ORDER nameJOBS 1ORACLE_COPIES 1PARALLEL_TABLES 1DEFAULT_PARALLELISM_DEGREE 0PARALLEL_MIN_ROWS 100000DROP_INDEXES 0SYNCHRONOUS_COMMIT 0EXPORT_INVALID 0PLSQL_PGSQL 1NULL_EQUAL_EMPTY 0EMPTY_LOB_NULL 0PACKAGE_AS_SCHEMA 1REWRITE_OUTER_JOIN 1FUNCTION_STABLE 1COMMENT_COMMIT_ROLLBACK 0COMMENT_SAVEPOINT 0USE_ORAFCE 0AUTONOMOUS_TRANSACTION 1ESTIMATE_COST 0COST_UNIT_VALUE 5DUMP_AS_HTML 0TOP_MAX 10HUMAN_DAYS_LIMIT 5PG_VERSION 11BITMAP_AS_GIN 1PG_BACKGROUND 0PG_SUPPORTS_SUBSTR 1AUTODETECT_SPATIAL_TYPE 1CONVERT_SRID 1DEFAULT_SRID 4326GEOMETRY_EXTRACT_TYPE INTERNALFDW_SERVER orclMYSQL_PIPES_AS_CONCAT 0MYSQL_INTERNAL_EXTRACT_FORMAT 0
2.6.2 创建导出数据配置文件
#cat etc/ora2pg/ora2pg.conf.dist | grep -v ^# |grep -v ^$ >ora2pg_data.conf#[root@test01 ora2pg]# ora2pg_data.confORACLE_HOME /u01/app/oracle/product/12.1.0/db_1ORACLE_DSN dbi:Oracle:host=192.168.0.9;sid=orcl;port=1521ORACLE_USER systemORACLE_PWD systemSCHEMA demoEXPORT_SCHEMA 1DISABLE_UNLOGGED 1SKIP fkeys ukeys checksTYPE COPYOUTPUT demo_data.sql
2.7 检查环境(可选)
检查软件环境:
[root@oracle ora2gp]# vim check.pl#!/usr/bin/perluse strict;use ExtUtils::Installed;my $inst= ExtUtils::Installed->new();my @modules = $inst->modules();foreach(@modules){ my $ver = $inst->version($_) || "???";printf("%-12s -- %s\n", $_, $ver);}exit;
2.8 ORACLE更新统计信息(可选)
更新oracle统计信息 提高性能,由dba去执行
BEGINDBMS_STATS.GATHER_SCHEMA_STATS('DEMO');DBMS_STATS.GATHER_DATABASE_STATS ;DBMS_STATS.GATHER_DICTIONARY_STATS;END;/
2.9 查询ORACLE源端对象
select object_type,count(object_type) from dba_objects where OWNER in ('DEMO')group by object_type;
2.10 测试连接(可选)
设置oracle数据库dsn后,可执行ora2pg以查看它是否有效:
[root@oracle ora2pg]# ora2pg -t show_version -c ora2pg.confOracle Database 12c Enterprise Edition Release 12.1.0.2.0[root@oracle ora2pg]#
2.11 迁移成本估算(重要)
估算从Oracle到PostgreSQL的迁移过程的成本并不容易。为了获得对此迁移成本的良好评估,Ora2Pg将检查所有数据库对象,所有函数和存储过程,以检测是否仍有一些对象和PL / SQL代码无法由Ora2Pg自动转换。
Ora2Pg具有内容分析模式,该模式检查Oracle数据库以生成有关Oracle数据库包含的内容和无法导出的内容的文本报告。
[root@oracle ora2pg]# ora2pg -t show_report --estimate_cost -c ora2pg_all.conf
estimate_cost(已过滤DB关键信息).txt
Object Number Invalid Estimated cost Comments Details-------------------------------------------------------------------------------DATABASE LINK 0 0 0 Database links will be exported as SQL/MEDPostgreSQL's Foreign Data Wrapper (FDW) extensions using oracle_fdw.FUNCTION 87 1 468.9 Total size of function code: 95246 bytes.GLOBAL TEMPORARY TABLE 59 0 168 Global temporary table are not supported byPostgreSQL and will not be exported. You will have to rewrite some applicationcode to match the PostgreSQL temporary table behavior.INDEX 1273 0 147.4 634 index(es) are concerned by the export, othersare automatically generated and will do so on PostgreSQL. Bitmap will beexported as btree_gin index(es) and hash index(es) will be exported as b-treeindex(es) if any. Domain index are exported as b-tree but commented to be edited tomainly use FTS. Cluster, bitmap join and IOT indexes will not be exported at all.Reverse indexes are not exported too, you may use a trigram-based index (see pg_trgm)or a reverse() function based index and search. Use 'varchar_pattern_ops', 'text_pattern_ops'or 'bpchar_pattern_ops' operators in your indexes to improve search with the LIKE operator respectivelyinto varchar, text or char columns. 2 bitmap index(es). 632 b-tree index(es).JAVA CLASS 1 1 0JAVA SOURCE 1 1 0JOB 0 0 0 Job are not exported. You may set external cron job with them.PACKAGE BODY 55 20 271.5 Total size of package code: 937271 bytes.Number of procedures and functions found inside those packages: 13.pp_test_yzh.p_rebuild_all_index: 7.5. pp_test_yzh.test_clear_gw: 4.1.pp_test_yzh.test_create_index: 6.8.pp_test_yzh.test_set_default: 4.7. pp_test_yzh.test_setblank_segment: 4.pp_判断数据类型.fu_isdate: 3.1. pp_判断数据类型.fu_isnumber: 3.1. pp_判断数据类型.fu_parsedate: 3.9.pp_判断数据类型.fu_pasnumber: 7. pp_判断数据类型.fu_pasenum_teshu: 4.7. pp_数据产出.fun_datadic: 4.5.pp_数据产出.fun_splitstr: 5. pp_档案管理.fu_age: 3.1.PROCEDURE 547 37 8973.90000000001 Total size of procedure code: 5564222 bytes.SEQUENCE 52 0 5.2 Sequences are fully supported,but all call to sequence_name.NEXTVAL or sequence_name.CURRVAL will be transformed into NEXTVAL('sequence_name') or CURRVAL('sequence_name').SYNONYM 1 0 1 SYNONYMs will be exported as views. SYNONYMs do not exists with PostgreSQLbut a common workaround is to use views or set the PostgreSQL search_path in your session to access object outside the current schema.TABLE 975 0 252 19 binary columns. 14 reserved words in column name. Total number of rows: 4000033001543.Top 10 of tables sorted by number of rows:TRIGGER 129 2 540.7 Total size of trigger code: 108222 bytes.TYPE 14 0 2 2 type(s) are concerned by the export, others are not supported.Note that Type inherited and Subtype are converted as table, type inheritance is not supported.3 associative arrays. 7 nested tables. 4 object type.VIEW 281 43 230.3 Views are fully supported but can use specific functions.inhospatientbaseinfo:-------------------------------------------------------------------------------Total 3475 105 11060.90 11060.90 cost migration units means approximatively 132 man-day(s).The migration unit was set to 5 minute(s)
通过迁移估算,可以看到3475个对象,失败为105个,成功率为97%,失败的内容会在迁移详情里面展示出内容。可以手动将失败的部分进行处理。迁移的工作量会大幅降低。
三、使用ora2pg
3.1导出全部表结构
如果在配置文件里指定的type包含了COPY,导完表结构后,表的数据也会随之导出,加-t参数即可只导出表结构不导出数据。
[root@oracle ~]#ora2pg -c ora2pg_table.conf[root@oracle ~]#ora2pg -c ora2pg_mview.conf[root@oracle ~]#ora2pg -c ora2pg_procedure.conf[root@oracle ~]#ora2pg -c ora2pg_sequence.conf[root@oracle ~]#ora2pg -c ora2pg_trigger.conf[root@oracle ~]#ora2pg -c ora2pg_type.conf[root@oracle ~]#ora2pg -c ora2pg_table.conf
分别生成sql脚本
-rw-r--r-- 1 root root 999045438 Jul 26 01:21 demo_data.sql-rw-r--r-- 1 root root 31 Jul 25 22:13 demo_mview.sql-rw-r--r-- 1 root root 7355369 Jul 26 02:11 demo_procedure.sql-rw-r--r-- 1 root root 5972 Jul 25 21:53 demo_sequence.sql-rw-r--r-- 1 root root 914559 Jul 25 22:22 demo_table.sql-rw-r--r-- 1 root root 226387 Jul 25 22:17 demo_trigger.sql-rw-r--r-- 1 root root 632162 Jul 25 21:56 demo_view.sql
3.2 通过ora2pg迁移数据
之前已创建好数据库以及相关表结构,现在只需通过ora2pg将数据同步到PostgreSQL数据库上即可。
[root@oracle ~]#ora2pg -c ora2pg_data.conf
四、导入结构和数据
在postgresql数据服务器导入数据库对象结构和数据,也可以在nivicat15导入脚本
#psql -U demo -d demo -a -f demo_table.sql#psql -U demo -d demo -a -f demo_mview.sql#psql -U demo -d demo -a -f demo_view.sql#psql -U demo -d demo -a -f demo_procedure.sql#psql -U demo -d demo -a -f demo_sequence.sql#psql -U demo -d demo -a -f demo_trigger.sql#psql -U demo -d demo -a -f demo_data.sql
导入完需要与oracle源核对数据对象数量和数据
五、迁移总结
先导出表结构,然后在同步数据; 在导出表结构的时候禁用分区表; 注意PostgreSQL的保留的关键字是否为oracle表字段名; 删除所有外键约束。 直接通过ora2pg迁移数据,不要将oracle表数据导出为csv格式,然后再导入PostgreSQL数据库。 有部分对象在pg中不存在,需要自己定义。例如:
psql:demo_table.sql:8015: ERROR: function uuid_generate_v4() does not existLINE 2: id varchar(50) NOT NULL DEFAULT uuid_generate_v4(),^HINT: No function matches the given name and argument types.You might need to add explicit type casts.-bash-4.2$
解决办法:
yum install -y install postgresql-contrib*cp /usr/share/pgsql/extension/uuid-ossp.control /usr/pgsql-12/share/extension重新CREATE EXTENSION IF NOT EXISTS "uuid-ossp"
I Love PG
关于我们
中国开源软件推进联盟PostgreSQL分会(简称:中国PG分会)于2017年成立,由国内多家PostgreSQL生态企业所共同发起,业务上接受工信部中国电子信息产业发展研究院指导。中国PG分会是一个非盈利行业协会组织。我们致力于在中国构建PostgreSQL产业生态,推动PostgreSQL产学研用发展。
技术文章精彩回顾 PostgreSQL学习的九层宝塔 PostgreSQL职业发展与学习攻略 2019,年度数据库舍 PostgreSQL 其谁? Postgres是最好的开源软件 PostgreSQL是世界上最好的数据库 从Oracle迁移到PostgreSQL的十大理由 从“非主流”到“潮流”,开源早已值得拥有 PG活动精彩回顾 创建PG全球生态!PostgresConf.CN2019大会盛大召开 首站起航!2019“让PG‘象’前行”上海站成功举行 走进蓉城丨2019“让PG‘象’前行”成都站成功举行 中国PG象牙塔计划发布,首批合作高校授牌仪式在天津举行 群英论道聚北京,共话PostgreSQL 相聚巴厘岛| PG Conf.Asia 2019 DAY0、DAY1简报 相知巴厘岛| PG Conf.Asia 2019 DAY2简报 独家|硅谷Postgres大会简报 直播回顾 | Bruce Momjian:原生分布式将在PG 14版本发布 PG培训认证精彩回顾 中国首批PGCA认证考试圆满结束,203位考生成功获得认证! 中国第二批PGCA认证考试圆满结束,115位考生喜获认证! 重要通知:三方共建,中国PostgreSQL认证权威升级! 近500人参与!首次PGCE中级、第三批次PGCA初级认证考试落幕! 2020年首批 | 中国PostgreSQL初级认证考试圆满结束 一分耕耘一分收获,第五批次PostgreSQL认证考试成绩公布





