暂无图片
暂无图片
10
暂无图片
暂无图片
暂无图片

数据库迁移神器推荐-SQLines

原创 肖杰 2021-09-06
7764

SQLines

SQLines provides tools to help you transfer data, convert database schema (DDL), views, stored procedures, packages, user-defined functions (UDFs), triggers, SQL queries, SQL scripts between different database platforms.

支持的迁移场景

Migration to MariaDB

IBM DB2 to MariaDB
MySQL to MariaDB
Oracle to MariaDB
SQL Server to MariaDB
Sybase ASE to MariaDB
Sybase ASA to MariaDB
Informix to MariaDB

Migration to Oracle

IBM DB2 to Oracle
Informix to Oracle
Sybase ASE to Oracle
Sybase ASA to Oracle
SQL Server to Oracle
Teradata to Oracle
PostgreSQL to Oracle
MySQL to Oracle

Oracle PL/SQL to Java
COBOL to Oracle PL/SQL

Migration to SQL Server

IBM DB2 to SQL Server
Informix to SQL Server
MySQL to SQL Server
Oracle to SQL Server
Sybase ASE to SQL Server
Sybase ASA to SQL Server
Sybase ADS to SQL Server

Migration to MySQL

IBM DB2 to MySQL
Informix to MySQL
Sybase ASE to MySQL
Oracle to MySQL
SQL Server to MySQL

Migration to PostgreSQL

IBM DB2 to PostgreSQL
Informix to PostgreSQL
Sybase ASE to PostgreSQL
Sybase ASA to PostgreSQL
Oracle to PostgreSQL
SQL Server to PostgreSQL

Migration to IBM DB2

SQL Server to IBM DB2

Migration to Redshift

Hadoop to Redshift
Oracle to Redshift
SQL Server to Redshift
Teradata to Redshift
Migration to Snowflake
IBM DB2 to Snowflake
Hadoop to Snowflake
Oracle to Snowflake
SQL Server to Snowflake
Teradata to Snowflake

Migration to Apache Hive

MySQL to Hive
Oracle to Hive
Redshift to Hive
SQL Server to Hive
Teradata to Hive

Migration to Spark

Oracle to Spark
Redshift to Spark
SQL Server to Spark
Teradata to Spark

Migration to Presto/Trino

Oracle to Presto/Trino
Redshift to Presto/Trino
SQL Server to Presto/Trino
Teradata to Presto/Trino

Migration to Netezza

Oracle to Netezza

Migration to Greenplum

IBM DB2 to Greenplum
Oracle to Greenplum

Migration to EsgynDB

Oracle to EsgynDB
Teradata to EsgynDB

官网

https://www.sqlines.com/

SQLines Data

SQLines Data is a high performance data transfer, schema conversion and migration validation tool that supports major enterprise databases:

  • Oracle
  • Microsoft SQL Server
  • MySQL
  • MariaDB
  • PostgreSQL
  • IBM DB2 LUW, iSeries (AS/400) and zSeries (S/390)
  • Sybase Adaptive Server Enterprise, Sybase SQL Anywhere, Sybase IQ and Sybase Advantage
  • Informix

You can use SQLines Data for cross-platform database migration. The tool migrates table definitions, constraints, indexes and transfers data.

Command Line Parameters

The following parameters are available:

Option Description Default Value
-sd Source database connection string Option must be specified, no default value
-td Target database connection string Option must be specified, no default value
-t List of tables (wildcards . are allowed) No default value
-tf A file with list of tables No file used
-texcl Tables to exclude (wildcards . are allowed) No tables excluded
-qf A file with SQL SELECT queries No file used
-cmd Command to perform Perform transfer command
-topt Transfer options recreate tables in the target database
-vopt Validation options rowcount to compare row count for tables
-smap Schema name mapping No any schema mapping is performed
-cmapf Column name and data type mapping file sqlines_cmap.txt file
-dtmapf Global data type mapping file sqlines_dtmap.txt file
-out Output directory for logs and reports (created if not exists) The current directory
-ss Number of concurrent sessions 4 concurrent sessions
-log Log file name (directory also can be specified) sqldata.log in the output directory
-cfg Configuration file name (directory also can be specified) sqldata.cfg in the current directory
-? Print to use the tool and exit

Note. Parameters are case-insensitive, and you can specify them in any order.

SQLines SQL Converter

SQLines SQL Converter converts database schema (DDL), queries and DML statements, views, stored procedures, packages, functions and triggers between all major databases.

Command Line Parameters

The following general parameters are available:

Option Description Default
-s Source database Option must be specified
-t Target database Option must be specified
-in List of files (wildcards . are allowed) Option must be specified unless -stdin is set
-stdin Read input from STDIN and send the result to STDOUT Read from -in file
-out Output file or directory
-log Log file path sqlines.log in the current directory
-rems Remove schema name in target scripts No
-smap Schema name mapping No mapping
-omapf Database object name mapping file No mapping
-meta Column meta data file No meta data
-a Create assessment report

-s and -t - Source and Target Database Types

-s and -t option allow you to specify the source and target database types (SQL dialects), respectively.

Use the following values:

Option Value Database
1 oracle Oracle
2 sql Microsoft SQL Server
3 db2 IBM DB2 for z/OS and LUW
4 sybase Sybase Adaptive Server Enterprise
5 asa Sybase Adaptive Server Anywhere, SQL Anywhere
6 mysql MySQL
7 postgresql PostgreSQL
8 informix Informix Dynamic Server, Extended Parallel Server
9 greenplum Greenplum
10 teradata Teradata
11 netezza Netezza
12 mariadb MariaDB
13 mariadb_ora MariaDB Oracle Compatibility mode
14 hive Apache Hive
15 redshift Amazon Redshift
16 snowflake Snowflake
17 spark Spark
18 trino Trino, Presto

For example, specify the following command line to convert script.sql file from MySQL to Oracle:

sqlines -s=mysql -t=oracle -in=script.sql

The result will be script_out.sql file in the current directory.

SQLines Data使用测试

Windows版:

  • sqldata:命令行工具,在CMD中调用
  • sqldataw:图形化工具

Linux版:

  • sqldata

配置文件解释

目录结构
[oracle@devin-enmo sqlines]$ ll
total 16060
-rwxr-xr-x 1 oracle oinstall  2701504 May 13 19:10 libQtCore.so
-rwxr-xr-x 1 oracle oinstall 11670344 May 13 19:10 libQtGui.so
-rw-r--r-- 1 oracle oinstall      118 May 13 19:10 license.txt
-rw-r--r-- 1 oracle oinstall      188 May 13 19:10 readme.txt
-rwxr-xr-x 1 oracle oinstall  2010797 May 13 19:10 sqldata
-rw-r--r-- 1 oracle oinstall     6575 May 12 17:48 sqldata.cfg
-rw-r--r-- 1 oracle oinstall      272 Sep  6 09:27 sqldata.log
-rw-r--r-- 1 oracle oinstall      401 May 13 19:10 sqlines_cmap.txt
-rw-r--r-- 1 oracle oinstall      202 May 13 19:10 sqlines_cnsmap.txt
-rw-r--r-- 1 oracle oinstall      190 May 13 19:10 sqlines_dtmap.txt
-rw-r--r-- 1 oracle oinstall      316 May 13 19:10 sqlines_qf.txt
-rw-r--r-- 1 oracle oinstall      292 Sep  6 09:15 sqlines_tf.txt
-rw-r--r-- 1 oracle oinstall      193 Sep  6 09:23 sqlines_tmap.txt
-rw-r--r-- 1 oracle oinstall      498 May 13 19:10 sqlines_tsel_all.txt
-rw-r--r-- 1 oracle oinstall      579 May 13 19:10 sqlines_tsel.txt
-rw-r--r-- 1 oracle oinstall      361 May 13 19:10 sqlines_twhere.txt
sqlines_cmap.txt
  • 作用:列名和数据类型映射文件
  • 语法:schema.table, column, target_column [, target_datatype]
  • 示例:SALES.CONTACTS, DESCRIPTION, DESC, VARCHAR2(4000)
sqlines_cnsmap.txt
  • 作用:表约束名映射文件
  • 语法:schema.table, source_constraint, target_constraint
  • 示例:SALES.CONTACTS, ID_UNIQUE_CNS, ID_SALES_UNIQUE
sqlines_dtmap.txt
  • 作用:全局数据类型映射文件

  • 语法:source_datatype(length, scale), target_datatype

  • 示例:

    – NUMERIC(*, 0), INTEGER
    – NUMBER(10,0), INT
    – VARCHAR2(4000), TEXT

sqlines_qf.txt
  • 作用:数据过滤,-qf=<file_name>
  • 语法:target_table1, query1;
  • 示例:ORDERS, SELECT * FROM orders WHERE created_dt >= CURRENT_DATE;
sqlines_tf.txt
  • 作用:指定需要传输的表的列表文件,-tf=<file_name>

  • 语法:schema.table1

  • 示例:

    ​ – dbo.ORDERS
    ​ – dbo.SALES

sqlines_tmap.txt
  • 作用:表名映射文件
  • 语法:schema.table, target_schema.target_table
  • 示例:SALES.CONTACTS, CRM.SALES_CONTACTS

还有一些其它的配置文件,不一一列出,每个配置文件里面都有详细解释及示例。

1,Oracle->PostgreSQL

基础环境

Oracle:

SQL> select count(*) from creditinfo_v; COUNT(*) ---------- 53343239 SQL> desc creditinfo_v; Name Null? Type ------------ --------- --------------------------- CI_EMAIL NOT NULL NVARCHAR2(100) CI_TEL NVARCHAR2(50) CI_FIRSTNAME NVARCHAR2(50) CI_LASTNAME NVARCHAR2(50) CI_COUNTRY NVARCHAR2(100) USERID NOT NULL NUMBER SQL> select sum(bytes)/1024/1024/1024 ||'GB' from dba_extents where segment_name='CREDITINFO_V'; SUM(BYTES)/1024/1024/1024||'GB' ------------------------------------------ 3.46051025390625GB --表大小:3.4G SQL> select index_name,table_name from dba_indexes where table_Name='CREDITINFO_V'; INDEX_NAME TABLE_NAME --------------- --------------- PK_CREDITINFO CREDITINFO_V IDX_CI_EMAIL CREDITINFO_V IDX_CI_TEL CREDITINFO_V IDX_1 CREDITINFO_V --表上一共有四个索引
执行迁移
--不同数据库的连接串写法及客户端配置,官网都有详细说明 [postgres@devin-enmo sqlines]$ ./sqldata -sd=oracle,ccps/xjlove1224@10.89.196.190:1521/hkora -td=pg,devin/xjlove1224@localhost:5432,devin -t=creditinfo_v -smap=ccps:devin -log=/home/postgres/sqlines.log SQLines Data 3.3.117 x86_64 Linux - Database Migration Tool. Copyright (c) 2021 SQLines. All Rights Reserved. FOR EVALUATION USE ONLY, MAY SKIP 1 ROW FROM A TABLE Connecting to databases (36 ms) Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production (Ok, 36 ms) PostgreSQL 13.2 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36.0.1), 64-bit (Ok, 4 ms) Reading the database schema (1 table, 10 ms) Transferring database (1 session): CCPS.CREDITINFO_V - Started (1 of 1, session 1) CCPS.CREDITINFO_V - Open cursor (10000 rows read, 24 ms, session 1)NOTICE: table "creditinfo_v" does not exist, skipping CCPS.CREDITINFO_V - Drop target table (1 ms, session 1) CCPS.CREDITINFO_V - Create target table (22 ms, session 1) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 1240000 (492846 rows/sec) Rows written: 1230000 (879199 rows/sec, 73.1 MB, 52.3 MB/sec) Transfer time: 3.1 sec (2.5 sec read, 1.4 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 2580000 (502336 rows/sec) Rows written: 2570000 (862127 rows/sec, 153.7 MB, 51.5 MB/sec) Transfer time: 6.3 sec (5.1 sec read, 3.0 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 3900000 (517447 rows/sec) Rows written: 3890000 (831730 rows/sec, 234.1 MB, 50.1 MB/sec) Transfer time: 9.4 sec (7.5 sec read, 4.7 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 5000000 (519157 rows/sec) Rows written: 4990000 (735554 rows/sec, 301.1 MB, 44.4 MB/sec) Transfer time: 12.8 sec (9.6 sec read, 6.8 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 6070000 (519558 rows/sec) Rows written: 6060000 (705636 rows/sec, 366.5 MB, 42.7 MB/sec) Transfer time: 15.9 sec (11.7 sec read, 8.6 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 6520000 (523653 rows/sec) Rows written: 6510000 (569653 rows/sec, 394.3 MB, 34.5 MB/sec) Transfer time: 19.2 sec (12.5 sec read, 11.4 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 7750000 (517702 rows/sec) Rows written: 7740000 (600978 rows/sec, 470.4 MB, 36.5 MB/sec) Transfer time: 22.3 sec (15.0 sec read, 12.9 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 8950000 (524496 rows/sec) Rows written: 8940000 (607915 rows/sec, 544.1 MB, 37.0 MB/sec) Transfer time: 25.4 sec (17.1 sec read, 14.7 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 10420000 (536975 rows/sec) Rows written: 10410000 (637594 rows/sec, 635.7 MB, 38.9 MB/sec) Transfer time: 28.5 sec (19.4 sec read, 16.3 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 11620000 (543372 rows/sec) Rows written: 11610000 (638087 rows/sec, 710.0 MB, 39.0 MB/sec) Transfer time: 31.6 sec (21.4 sec read, 18.2 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 12890000 (552697 rows/sec) Rows written: 12880000 (640064 rows/sec, 788.7 MB, 39.2 MB/sec) Transfer time: 34.7 sec (23.3 sec read, 20.1 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 13290000 (555486 rows/sec) Rows written: 13280000 (581309 rows/sec, 813.5 MB, 35.6 MB/sec) Transfer time: 37.8 sec (23.9 sec read, 22.8 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 14570000 (563027 rows/sec) Rows written: 14560000 (589593 rows/sec, 892.4 MB, 36.1 MB/sec) Transfer time: 41.0 sec (25.9 sec read, 24.7 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 16160000 (566441 rows/sec) Rows written: 16150000 (619843 rows/sec, 990.7 MB, 38.0 MB/sec) Transfer time: 44.1 sec (28.5 sec read, 26.1 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 17870000 (573418 rows/sec) Rows written: 17860000 (650638 rows/sec, 1.1 GB, 39.9 MB/sec) Transfer time: 47.2 sec (31.2 sec read, 27.4 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 19400000 (576146 rows/sec) Rows written: 19390000 (670007 rows/sec, 1.2 GB, 41.1 MB/sec) Transfer time: 50.3 sec (33.7 sec read, 28.9 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 20480000 (578449 rows/sec) Rows written: 20470000 (636465 rows/sec, 1.2 GB, 39.1 MB/sec) Transfer time: 54.6 sec (35.4 sec read, 32.2 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 22130000 (581848 rows/sec) Rows written: 22120000 (659118 rows/sec, 1.3 GB, 40.5 MB/sec) Transfer time: 57.8 sec (38.0 sec read, 33.6 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 23380000 (581954 rows/sec) Rows written: 23370000 (654274 rows/sec, 1.4 GB, 40.2 MB/sec) Transfer time: 1 min 1 sec (40.2 sec read, 35.7 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 24790000 (581202 rows/sec) Rows written: 24780000 (666936 rows/sec, 1.5 GB, 41.0 MB/sec) Transfer time: 1 min 4 sec (42.7 sec read, 37.2 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 26300000 (580856 rows/sec) Rows written: 26290000 (683550 rows/sec, 1.6 GB, 42.1 MB/sec) Transfer time: 1 min 7 sec (45.3 sec read, 38.5 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 27730000 (578951 rows/sec) Rows written: 27720000 (696133 rows/sec, 1.7 GB, 42.8 MB/sec) Transfer time: 1 min 10 sec (47.9 sec read, 39.8 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 29020000 (579959 rows/sec) Rows written: 29010000 (698767 rows/sec, 1.7 GB, 43.0 MB/sec) Transfer time: 1 min 13 sec (50.0 sec read, 41.5 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 29690000 (580336 rows/sec) Rows written: 29680000 (676051 rows/sec, 1.8 GB, 41.7 MB/sec) Transfer time: 1 min 16 sec (51.2 sec read, 43.9 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 31240000 (580356 rows/sec) Rows written: 31230000 (690471 rows/sec, 1.9 GB, 42.6 MB/sec) Transfer time: 1 min 20 sec (53.8 sec read, 45.2 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 32460000 (579840 rows/sec) Rows written: 32450000 (684816 rows/sec, 2.0 GB, 42.3 MB/sec) Transfer time: 1 min 23 sec (56.0 sec read, 47.4 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 33820000 (576720 rows/sec) Rows written: 33810000 (693880 rows/sec, 2.0 GB, 42.8 MB/sec) Transfer time: 1 min 26 sec (58.6 sec read, 48.7 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 35030000 (570958 rows/sec) Rows written: 35020000 (700274 rows/sec, 2.1 GB, 43.2 MB/sec) Transfer time: 1 min 29 sec (1 min 1 sec read, 50.0 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 36360000 (567797 rows/sec) Rows written: 36350000 (709061 rows/sec, 2.2 GB, 43.8 MB/sec) Transfer time: 1 min 32 sec (1 min 4 sec read, 51.3 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 37670000 (566577 rows/sec) Rows written: 37660000 (713636 rows/sec, 2.3 GB, 44.1 MB/sec) Transfer time: 1 min 36 sec (1 min 6 sec read, 52.8 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 38370000 (565746 rows/sec) Rows written: 38360000 (690003 rows/sec, 2.3 GB, 42.6 MB/sec) Transfer time: 1 min 39 sec (1 min 7 sec read, 55.6 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 39910000 (566767 rows/sec) Rows written: 39900000 (700332 rows/sec, 2.4 GB, 43.3 MB/sec) Transfer time: 1 min 42 sec (1 min 10 sec read, 57.0 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 41390000 (566242 rows/sec) Rows written: 41380000 (709765 rows/sec, 2.5 GB, 43.9 MB/sec) Transfer time: 1 min 45 sec (1 min 13 sec read, 58.3 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 42230000 (565025 rows/sec) Rows written: 42220000 (700119 rows/sec, 2.6 GB, 43.3 MB/sec) Transfer time: 1 min 49 sec (1 min 14 sec read, 1 min 0 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 43690000 (564492 rows/sec) Rows written: 43680000 (708252 rows/sec, 2.6 GB, 43.8 MB/sec) Transfer time: 1 min 52 sec (1 min 17 sec read, 1 min 1 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 45110000 (563375 rows/sec) Rows written: 45100000 (716044 rows/sec, 2.7 GB, 44.3 MB/sec) Transfer time: 1 min 55 sec (1 min 20 sec read, 1 min 2 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 46320000 (563483 rows/sec) Rows written: 46310000 (716263 rows/sec, 2.8 GB, 44.3 MB/sec) Transfer time: 1 min 58 sec (1 min 22 sec read, 1 min 4 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 47470000 (561862 rows/sec) Rows written: 47460000 (703007 rows/sec, 2.9 GB, 43.5 MB/sec) Transfer time: 2 min 2 sec (1 min 24 sec read, 1 min 7 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 49060000 (563067 rows/sec) Rows written: 49050000 (711829 rows/sec, 3.0 GB, 44.0 MB/sec) Transfer time: 2 min 5 sec (1 min 27 sec read, 1 min 8 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 50640000 (564033 rows/sec) Rows written: 50630000 (720701 rows/sec, 3.1 GB, 44.6 MB/sec) Transfer time: 2 min 8 sec (1 min 29 sec read, 1 min 10 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 51470000 (564624 rows/sec) Rows written: 51460000 (709715 rows/sec, 3.1 GB, 43.9 MB/sec) Transfer time: 2 min 12 sec (1 min 31 sec read, 1 min 12 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 53030000 (565334 rows/sec) Rows written: 53020000 (717728 rows/sec, 3.2 GB, 44.4 MB/sec) Transfer time: 2 min 15 sec (1 min 33 sec read, 1 min 13 sec write) CCPS.CREDITINFO_V - Data transfer complete (session 1) Rows read: 53343239 (565640 rows/sec) Rows written: 53343239 (719464 rows/sec, 3.2 GB, 44.6 MB/sec) Transfer time: 2 min 15 sec (1 min 34 sec read, 1 min 14 sec write) CCPS.CREDITINFO_V - Add DEFAULT clause (Failed, 2 ms, session 1) ERROR: cannot use column reference in DEFAULT expression CCPS.CREDITINFO_V - Add PRIMARY KEY constraint (1 min 7 sec, session 1) CCPS.CREDITINFO_V - Create index IDX_CI_EMAIL (4 min 10 sec, session 1) CCPS.CREDITINFO_V - Create index IDX_CI_TEL (3 min 27 sec, session 1) CCPS.CREDITINFO_V - Create index IDX_1 (1 min 2 sec, session 1) Summary: Tables: 1 (1 Ok, 0 failed) Target DDL: 7 (6 Ok, 1 failed) Rows read: 53343239 Rows written: 53343239 Transfer time: 12 min 4 sec (73646 rows/sec, 3.2 GB, 4.6 MB/sec) Logs: Execution log: /home/postgres/sqlines.log DDL SQL statements: sqldata_ddl.sql Failed DDL SQL statements: sqldata_failed.sql
迁移总结
  • 迁移表数量:1

  • DDL数量:7,6成功,1失败(失败具体内容可以根据提示查看sqldata_failed.sql)

    [postgres@devin-enmo sqlines]$ vi sqldata_failed.sql -- SQLines Data 3.3.117 x86_64 Linux - Database Migration Tool. -- Copyright (c) 2021 SQLines. All Rights Reserved. -- Failed DDL SQL statements executed for the target database -- Current timestamp: 2021:09:06 10:05:09.216 ALTER TABLE devin.CREDITINFO_V ALTER USERID SET DEFAULT "CCPS"."SEQ_CREDITINFO"."NEXTVAL"; -- Failed (2 ms) -- ERROR: cannot use column reference in DEFAULT expression ----可以看到,是因为的pg库没有创建相关序列导致
  • 数据行:53343239

  • 耗时:12分4秒,其中大部分时间都是创建索引,数据迁移只用了1分34秒

验证
[postgres@devin-enmo sqlines]$ psql -U devin -d devin psql (13.2) Type "help" for help. devin=> \d List of relations Schema | Name | Type | Owner --------+--------------+-------+------- devin | creditinfo_v | table | devin devin | devin1 | table | devin devin | test1 | table | devin (3 rows) devin=> select count(*) from creditinfo_v; count ---------- 53343239 (1 row)

迁移完成。

2,Oracle->MySQL

注:MySQL需要启用local_infile,否则报错

Loading local data is disabled
执行迁移
[oracle@devin-enmo sqlines]$ ./sqldata -sd=oracle,ccps/xjlove1224@10.89.196.190:1521/hkora -td=mysql,root/xjlove1224@localhost:3306,devin -t=creditinfo_v -smap=ccps:devin -log=/home/oracle/sqlines/sqlines.log SQLines Data 3.3.117 x86_64 Linux - Database Migration Tool. Copyright (c) 2021 SQLines. All Rights Reserved. FOR EVALUATION USE ONLY, MAY SKIP 1 ROW FROM A TABLE Connecting to databases (29 ms) Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production (Ok, 29 ms) MySQL 8.0.25 MySQL Community Server - GPL x86_64 Linux 1.2.11 (Ok, 5 ms) Reading the database schema (1 table, 9 ms) Transferring database (1 session): CCPS.CREDITINFO_V - Started (1 of 1, session 1) CCPS.CREDITINFO_V - Open cursor (10000 rows read, 20 ms, session 1) CCPS.CREDITINFO_V - Drop target table (27 ms, session 1) CCPS.CREDITINFO_V - Create target table (38 ms, session 1) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 580000 (643016 rows/sec) Rows written: 569999 (201271 rows/sec, 33.8 MB, 11.9 MB/sec) Transfer time: 3.1 sec (902 ms read, 2.8 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 770000 (647603 rows/sec) Rows written: 759999 (122957 rows/sec, 45.0 MB, 7.3 MB/sec) Transfer time: 6.5 sec (1.2 sec read, 6.2 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 970000 (639420 rows/sec) Rows written: 959999 (98179 rows/sec, 56.9 MB, 5.8 MB/sec) Transfer time: 10.2 sec (1.5 sec read, 9.8 sec write) CCPS.CREDITINFO_V - In progress (session 1) Rows read: 1220000 (625320 rows/sec) Rows written: 1209999 (94237 rows/sec, 71.9 MB, 5.6 MB/sec) Transfer time: 13.3 sec (2.0 sec read, 12.8 sec write) . . . . CCPS.CREDITINFO_V - Data transfer complete (session 1) Rows read: 53343239 (529966 rows/sec) Rows written: 53343238 (61221 rows/sec, 3.2 GB, 3.8 MB/sec) Transfer time: 14 min 49 sec (1 min 40 sec read, 14 min 31 sec write) CCPS.CREDITINFO_V - Add PRIMARY KEY constraint (16 min 51 sec, session 1) CCPS.CREDITINFO_V - Create index IDX_CI_EMAIL (8 min 23 sec, session 1) CCPS.CREDITINFO_V - Create index IDX_CI_TEL (5 min 17 sec, session 1) CCPS.CREDITINFO_V - Create index IDX_1 (3 min 43 sec, session 1) Read/write row count differences (1 table): 1. CCPS.CREDITINFO_V (53343239 rows read, 53343238 rows written, 1 rows difference) Please contact us at support@sqlines.com for any assistance. Summary: Tables: 1 (1 Ok, 0 failed) Target DDL: 6 (6 Ok, 0 failed) Rows read: 53343239 Rows written: 53343238 (1 row difference) Transfer time: 49 min 5 sec (18110 rows/sec, 3.2 GB, 1.1 MB/sec) Logs: Execution log: /home/oracle/sqlines/sqlines.log DDL SQL statements: sqldata_ddl.sql
迁移总结

Transfer time: 49 min 5 sec (18110 rows/sec, 3.2 GB, 1.1 MB/sec)

对比PG:

Transfer time: 12 min 4 sec (73646 rows/sec, 3.2 GB, 4.6 MB/sec)

可见mysql的性能是要差很多,可能与性能参数配置相关,本次测试都是在同一台机器不同用户,数据库均使用默认参数。

验证
[mysql@devin-enmo ~]$ mysql -uroot -P3306 -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 14 Server version: 8.0.25 MySQL Community Server - GPL Copyright (c) 2000, 2021, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> use devin; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +-----------------+ | Tables_in_devin | +-----------------+ | CREDITINFO_V | +-----------------+ 1 row in set (0.00 sec) mysql> select count(*) from CREDITINFO_V; +----------+ | count(*) | +----------+ | 53343238 | +----------+ 1 row in set (8.40 sec)

迁移完成。

3,图形化界面迁移数据

配置环境变量

配置数据库

1.png

配置需要迁移的表

2.png

开始迁移

3.png

迁移完成

4.png

看到有一条ddl执行失败,在sqlines解压目录下即可看到具体失败内容

微信截图_20210906182327.png

验证
devin=> select count(*) from creditinfo_v; count ---------- 53343238 (1 row)

迁移完成。

最后修改时间:2021-09-08 14:26:36
「喜欢这篇文章,您的关注和赞赏是给作者最好的鼓励」
关注作者
【版权声明】本文为墨天轮用户原创内容,转载时必须标注文章的来源(墨天轮),文章链接,文章作者等基本信息,否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容,欢迎发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

评论