暂无图片
暂无图片
1
暂无图片
暂无图片
暂无图片
首页 / GPU数据库HeavyDB初体验--OLAP轻松以一敌百

GPU数据库HeavyDB初体验--OLAP轻松以一敌百

原创 jieguo 2022-05-08
2805

image.png
参考文档：https://github.com/heavyai/heavydb
HeavyDB (formerly OmniSciDB)
HeavyDB是一个开源的基于SQL的关系型列数据库引擎，它利用现代硬件(CPU和GPU)的全部性能和并行性，能够在几毫秒内查询数十亿行数据集，而无需索引、预聚合或缩减采样。HeavyDB可以在混合CPU/GPU系统(目前支持Nvidia GPUs)上运行，也可以在以X86、Power和ARM(实验支持)架构为特色的纯CPU系统上运行。为了实现最高的性能，HeavyDB具有存储、CPU内存和GPU内存之间的多层数据缓存，以及创新的实时(JIT)查询编译框架。

安装步骤：(由于笔记本虚机无法使用gpu,所以先安装个cpu版本的体验下，充分利用内存，速度的确惊人)
曾经在1块11G显存的GPU主机测试17亿记录只需要79ms
image.png
image.png
1.操作系统centos7.9最小化安装http://mirrors.aliyun.com/centos/7.9.2009/isos/x86_64/CentOS-7-x86_64-DVD-2009.iso
2.heavydb下载：https://releases.heavy.ai/os/tar/heavyai-os-latest-Linux-x86_64-cpu.tar.gz
如果是gpu环境，下载：https://releases.heavy.ai/os/tar/heavyai-os-latest-Linux-x86_64.tar.gz
3.安装依赖包：
yum install wget net-tools -y
wget https://github.com/heavyai/heavydb/blob/master/scripts/mapd-deps-prebuilt.sh获取脚本执行安装依赖包，执行到如下截图位置即可取消。
image.png
4.开始安装：
解压tar -zxvf omnisci-os-latest-Linux-x86_64-cpu.tar.gz
建数据目录：mkdir data
初始化数据库：
./bin/initdb data
启动数据库服务：
nohup ./bin/omnisci_server > ./omnisci.log &
netstat -tunlp|grep LISTEN
image.png
插入测试数据：
…/insert_sample_data
命令行登录管理：
./bin/omnisql -p HyperInteractive
where **HyperInteractive **is the default password. The default user admin is assumed if not provided.
操作记录：

[root@heavydb ~]# cd cpu20220218/
[root@heavydb cpu20220218]# ll
total 68
drwxrwxr-x. 2 1004 1005   240 Feb 18 23:48 bin
-rw-r--r--. 1 1004 1005   653 Feb 18 23:07 common.thrift
-rw-r--r--. 1 1004 1005   306 Feb 18 23:07 completion_hints.thrift
drwxr-xr-x. 7 root root   135 May  8 18:24 data
drwxrwxr-x. 2 1004 1005    41 Feb 18 23:48 docker
-rwxr-xr-x. 1 1004 1005  2702 Feb 18 23:07 insert_sample_data
-rw-r--r--. 1 1004 1005 11383 Feb 18 23:07 LICENSE.md
drwxrwxr-x. 2 1004 1005    22 Feb 18 23:48 Logger
-rwxr-xr-x. 1 root root  7143 May  8 18:16 mapd-deps-prebuilt.sh
-rw-r--r--. 1 1004 1005    11 Feb 18 23:47 MAPD_GIT_HASH.txt
-rw-r--r--. 1 1004 1005 23456 Feb 18 23:07 omnisci.thrift
drwxrwxr-x. 2 1004 1005   177 Feb 18 23:48 QueryEngine
drwxr-xr-x. 3 1004 1005  4096 Feb 18 23:48 SampleCode
drwxrwxr-x. 2 1004 1005    26 Feb 18 23:48 scripts
drwxrwxr-x. 2 1004 1005    57 Feb 18 23:48 Shared
-rwxr-xr-x. 1 1004 1005  2700 Feb 18 23:07 startomnisci
drwxrwxr-x. 2 1004 1005   130 Feb 18 23:48 systemd
drwxrwxr-x. 5 1004 1005    58 Feb 18 23:48 ThirdParty
[root@heavydb cpu20220218]# ./insert_sample_data 
Using default data directory: "/root/cpu20220218/data" if file path is not whitelisted use '--data /path_to_server_data_directory'
~/cpu20220218/data/mapd_import/sample_datasets ~/cpu20220218
--2022-05-08 18:26:36--  https://data.mapd.com/manifest.tsv
Resolving data.mapd.com (data.mapd.com)... 72.28.97.165
Connecting to data.mapd.com (data.mapd.com)|72.28.97.165|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 192 [application/octet-stream]
Saving to: 鈥榤anifest.tsv鈥

100%[===================================================================================================================>] 192         --.-K/s   in 0s      

2022-05-08 18:26:37 (8.69 MB/s) - 鈥榤anifest.tsv鈥saved [192/192]

~/cpu20220218
Enter dataset number to download, or 'q' to quit:
 #     Dataset                   Rows    Table Name             File Name
 1)    Flights (2008)            7M      flights_2008_7M        flights_2008_7M.tar.gz
 2)    Flights (2008)            10k     flights_2008_10k       flights_2008_10k.tar.gz
 3)    NYC Tree Census (2015)    683k    nyc_trees_2015_683k    nyc_trees_2015_683k.tar.gz
2
~/cpu20220218/data/mapd_import/sample_datasets ~/cpu20220218
- downloading and extracting flights_2008_10k.tar.gz
--2022-05-08 18:26:39--  https://data.mapd.com/flights_2008_10k.tar.gz
Resolving data.mapd.com (data.mapd.com)... 72.28.97.165
Connecting to data.mapd.com (data.mapd.com)|72.28.97.165|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 584277 (571K) [application/octet-stream]
Saving to: 鈥榝lights_2008_10k.tar.gz鈥

100%[===================================================================================================================>] 584,277      241KB/s   in 2.4s   

2022-05-08 18:26:43 (241 KB/s) - 鈥榝lights_2008_10k.tar.gz鈥saved [584277/584277]

flights_2008_10k/
flights_2008_10k/flights_2008_10k.csv
flights_2008_10k/flights_2008_10k.sql
~/cpu20220218
- adding schema
Thrift: Sun May  8 18:26:43 2022 TSocket::open() connect() <Host: localhost Port: 6274>: Connection refused
User admin connected to database omnisci
User admin disconnected from database omnisci
- inserting file: /root/cpu20220218/data/mapd_import/sample_datasets/flights_2008_10k/flights_2008_10k.csv
Thrift: Sun May  8 18:26:43 2022 TSocket::open() connect() <Host: localhost Port: 6274>: Connection refused
User admin connected to database omnisci
Result
Loaded: 10000 recs, Rejected: 0 recs in 0.381000 secs
User admin disconnected from database omnisci
[root@heavydb cpu20220218]# ll
total 68
drwxrwxr-x. 2 1004 1005   240 Feb 18 23:48 bin
-rw-r--r--. 1 1004 1005   653 Feb 18 23:07 common.thrift
-rw-r--r--. 1 1004 1005   306 Feb 18 23:07 completion_hints.thrift
drwxr-xr-x. 8 root root   154 May  8 18:26 data
drwxrwxr-x. 2 1004 1005    41 Feb 18 23:48 docker
-rwxr-xr-x. 1 1004 1005  2702 Feb 18 23:07 insert_sample_data
-rw-r--r--. 1 1004 1005 11383 Feb 18 23:07 LICENSE.md
drwxrwxr-x. 2 1004 1005    22 Feb 18 23:48 Logger
-rwxr-xr-x. 1 root root  7143 May  8 18:16 mapd-deps-prebuilt.sh
-rw-r--r--. 1 1004 1005    11 Feb 18 23:47 MAPD_GIT_HASH.txt
-rw-r--r--. 1 1004 1005 23456 Feb 18 23:07 omnisci.thrift
drwxrwxr-x. 2 1004 1005   177 Feb 18 23:48 QueryEngine
drwxr-xr-x. 3 1004 1005  4096 Feb 18 23:48 SampleCode
drwxrwxr-x. 2 1004 1005    26 Feb 18 23:48 scripts
drwxrwxr-x. 2 1004 1005    57 Feb 18 23:48 Shared
-rwxr-xr-x. 1 1004 1005  2700 Feb 18 23:07 startomnisci
drwxrwxr-x. 2 1004 1005   130 Feb 18 23:48 systemd
drwxrwxr-x. 5 1004 1005    58 Feb 18 23:48 ThirdParty
[root@heavydb cpu20220218]# cd bin
[root@heavydb bin]# ll
total 418344
-rw-r--r--. 1 1004 1005  33758974 Feb 18 23:09 calcite-1.0-SNAPSHOT-jar-with-dependencies.jar
-rwxr-xr-x. 1 1004 1005   3370621 Feb 18 23:08 generate_cert
-rwxr-xr-x. 1 1004 1005 159078688 Feb 18 23:48 initdb
-rwxr-xr-x. 1 1004 1005  10004672 Feb 18 23:48 KafkaImporter
-rw-r--r--. 1 1004 1005   6550078 Feb 18 23:08 omnisci-jdbc-5.10.2.jar
-rwxr-xr-x. 1 1004 1005 161795152 Feb 18 23:48 omnisci_server
-rw-r--r--. 1 1004 1005  36229199 Feb 18 23:09 omnisci-utility-5.10.2.jar
-rwxr-xr-x. 1 1004 1005   8910904 Feb 18 23:48 omnisql
-rwxr-xr-x. 1 1004 1005   8668088 Feb 18 23:48 StreamImporter
[root@heavydb bin]# ./omnisql
Password: 
Thrift: Sun May  8 18:29:51 2022 TSocket::open() connect() <Host: localhost Port: 6274>: Connection refused
User admin connected to database omnisci
omnisql> \l
Database | Owner
omnisci | admin
omnisql> \t
omnisci_states
omnisci_counties
omnisci_countries
flights_2008_10k
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
     6242.23 MB        0.00 MB        0.00 MB        0.00 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE

omnisql> select count(*) from flights_2008_10k;
EXPR$0
10000
omnisql> \timing
omnisql> select count(*) from flights_2008_10k;
EXPR$0
10000
1 rows returned.
Execution time: 21 ms, Total time: 21 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
     6242.23 MB        0.00 MB        0.00 MB        0.00 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE

omnisql> \q
User admin disconnected from database omnisci

6000多万条记录，count只需17毫秒；
image.png
image.png
1.2亿数据，count在第二次执行只需20毫秒：
image.png
image.png

[root@heavydb cpu20220218]# ./insert_sample_data 
Using default data directory: "/root/cpu20220218/data" if file path is not whitelisted use '--data /path_to_server_data_directory'
~/cpu20220218/data/mapd_import/sample_datasets ~/cpu20220218
--2022-05-08 18:30:55--  https://data.mapd.com/manifest.tsv
Resolving data.mapd.com (data.mapd.com)... 72.28.97.165
Connecting to data.mapd.com (data.mapd.com)|72.28.97.165|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 192 [application/octet-stream]
Saving to: 鈥榤anifest.tsv鈥

100%[===================================================================================================================>] 192         --.-K/s   in 0s      

2022-05-08 18:30:56 (39.9 MB/s) - 鈥榤anifest.tsv鈥saved [192/192]

~/cpu20220218
Enter dataset number to download, or 'q' to quit:
 #     Dataset                   Rows    Table Name             File Name
 1)    Flights (2008)            7M      flights_2008_7M        flights_2008_7M.tar.gz
 2)    Flights (2008)            10k     flights_2008_10k       flights_2008_10k.tar.gz
 3)    NYC Tree Census (2015)    683k    nyc_trees_2015_683k    nyc_trees_2015_683k.tar.gz
1
~/cpu20220218/data/mapd_import/sample_datasets ~/cpu20220218
- downloading and extracting flights_2008_7M.tar.gz
--2022-05-08 18:31:00--  https://data.mapd.com/flights_2008_7M.tar.gz
Resolving data.mapd.com (data.mapd.com)... 72.28.97.165
Connecting to data.mapd.com (data.mapd.com)|72.28.97.165|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 377039522 (360M) [application/octet-stream]
Saving to: 鈥榝lights_2008_7M.tar.gz鈥

61% [======================================================================>                                             ] 231,808,767  224KB/s  eta 9m 37s ^100%[===================================================================================================================>] 377,039,522  370KB/s   in 23m 9s 

2022-05-08 18:54:12 (265 KB/s) - 鈥榝lights_2008_7M.tar.gz鈥saved [377039522/377039522]

flights_2008_7M/
flights_2008_7M/flights_2008_7M.csv
flights_2008_7M/flights_2008_7M.sql
~/cpu20220218
- adding schema
Thrift: Sun May  8 18:54:28 2022 TSocket::open() connect() <Host: localhost Port: 6274>: Connection refused
User admin connected to database omnisci
User admin disconnected from database omnisci
- inserting file: /root/cpu20220218/data/mapd_import/sample_datasets/flights_2008_7M/flights_2008_7M.csv
Thrift: Sun May  8 18:54:28 2022 TSocket::open() connect() <Host: localhost Port: 6274>: Connection refused
User admin connected to database omnisci
Result
Loaded: 7009728 recs, Rejected: 0 recs in 10.020000 secs
User admin disconnected from database omnisci
[root@heavydb cpu20220218]# ./bin/omnisql -p HyperInteractive
Thrift: Sun May  8 18:56:03 2022 TSocket::open() connect() <Host: localhost Port: 6274>: Connection refused
User admin connected to database omnisci
omnisql> \l
Database | Owner
omnisci | admin
omnisql> \t
omnisci_states
omnisci_counties
omnisci_countries
flights_2008_10k
flights_2008_7M
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
     6242.23 MB        0.00 MB        0.00 MB        0.00 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE

omnisql> select count(*) from flights_2008_7M;
EXPR$0
7009728
omnisql> \timing
omnisql> select count(*) from flights_2008_7M;
EXPR$0
7009728
1 rows returned.
Execution time: 25 ms, Total time: 25 ms
omnisql> \cpu
omnisql> select count(*) from flights_2008_7M;
EXPR$0
7009728
1 rows returned.
Execution time: 10 ms, Total time: 11 ms
omnisql> select count(*) from flights_2008_7M;
EXPR$0
7009728
1 rows returned.
Execution time: 11 ms, Total time: 11 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
     6242.23 MB        0.00 MB        0.00 MB        0.00 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE

omnisql> insert into flights_2008_7M select * from flights_2008_7M;
omnisql> select count(*) from flights_2008_7M;
EXPR$0
15009728
1 rows returned.
Execution time: 11 ms, Total time: 12 ms
omnisql> insert into flights_2008_7M select * from flights_2008_7M;
omnisql> insert into flights_2008_7M select * from flights_2008_7M;
omnisql> select count(*) from flights_2008_7M;
EXPR$0
63009728
1 rows returned.
Execution time: 17 ms, Total time: 18 ms
omnisql> select count(*) from flights_2008_7M;
EXPR$0
63009728
1 rows returned.
Execution time: 14 ms, Total time: 15 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
     6242.23 MB     5554.30 MB     6242.23 MB      687.94 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE

omnisql> select count(*) from flights_2008_7M;
EXPR$0
63009728
1 rows returned.
Execution time: 16 ms, Total time: 17 ms

思考下：gpu上万个核心的优势是cpu无可比拟的，在数据库分布式，MPP之后，将来数据库趋势何去何从？cpu+gpu组合，非常看好GPU，立贴记录。。。

经测试发现：物理内存默认最大可占用80%，GPU显卡内存最大占用95%。数据加载会先load到物理内存，在按条件查询时会load到gpu加速查询结果，每次4G递增分配。
----------gpu下测试------------
omnisky@omnisky-Super-Server:~/gpudb$ nvidia-smi -a|grep Product
Product Name : NVIDIA GeForce RTX 2080 Ti
Product Brand : GeForce
Product Architecture : Turing

omnisql> select count(*) from nyc_trees_2015_683k where tree_id='250990';
EXPR$0
355
1 rows returned.
Execution time: 27 ms, Total time: 29 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB    26959.74 MB    28672.00 MB     1712.26 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB     2215.21 MB     4096.00 MB     1880.79 MB

omnisql> select count(*) from nyc_trees_2015_683k where tree_id='250991';
EXPR$0
352
1 rows returned.
Execution time: 33 ms, Total time: 35 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB    26959.74 MB    28672.00 MB     1712.26 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB     2215.21 MB     4096.00 MB     1880.79 MB

omnisql> select * from nyc_trees_2015_683k where tree_id='250991' limit 10;
created_at|tree_id|block_id|the_geom|tree_dbh|stump_diam|curb_loc|status|health|spc_latin|spc_common|steward|guards|sidewalk|user_type|problems|root_stone|root_grate|root_other|trnk_wire|trnk_light|trnk_other|brnch_ligh|brnch_shoe|brnch_othe|address|zipcode|zip_city|cb_num|borocode|boroname|cncldist|st_assem|st_senate|nta|nta_name|boro_ct|state|latitude|longitude|x_sp|y_sp
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
2015-09-23|250991|201178|POINT (-73.89310741903807 40.669108933880445)|10|0|OnCurb|Alive|Good|Tilia americana|American linden|1or2|None|NoDamage|NYC Parks Staff|BranchLights|No|No|No|No|No|No|Yes|No|No|429 VERMONT STREET|11207|Brooklyn|305|3|Brooklyn|42|60|19|BK85|East New York (Pennsylvania Ave)|3115800|New York|40.66911|-73.8931|1013903|183068.9
10 rows returned.
Execution time: 68 ms, Total time: 73 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB    26959.74 MB    28672.00 MB     1712.26 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB     2215.21 MB     4096.00 MB     1880.79 MB

omnisql> \q
User admin disconnected from database omnisci
omnisky@omnisky-Super-Server:~/gpudb$ free -m
              total        used        free      shared  buff/cache   available
Mem:         257825       32434       81014         425      144376      222949
Swap:          2047           0        2047
omnisky@omnisky-Super-Server:~/gpudb$ nvidia-smi 
Tue May 10 01:51:21 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02    Driver Version: 510.60.02    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:03:00.0  On |                  N/A |
| 30%   40C    P8    29W / 250W |   4494MiB / 11264MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      2508      G   /usr/lib/xorg/Xorg                147MiB |
|    0   N/A  N/A      2958      G   /usr/bin/gnome-shell               55MiB |
|    0   N/A  N/A     11465      G   ...nlogin/bin/sunloginclient        5MiB |
|    0   N/A  N/A     31699      C   ./bin/omnisci_server             4279MiB |
+-----------------------------------------------------------------------------+
omnisky@omnisky-Super-Server:~/gpudb$ ./bin/omnisql -p HyperInteractive
User admin connected to database omnisci
omnisql> select count(*) from nyc_trees_2015_683k;
EXPR$0
235762432
omnisql> \gpu
omnisql> \timing
omnisql> select count(*) from nyc_trees_2015_683k;
EXPR$0
235762432
1 rows returned.
Execution time: 22 ms, Total time: 24 ms
omnisql> select count(*) from nyc_trees_2015_683k;
EXPR$0
235762432
1 rows returned.
Execution time: 20 ms, Total time: 21 ms
omnisql> select count(*) from nyc_trees_2015_683k;
EXPR$0
235762432
1 rows returned.
Execution time: 18 ms, Total time: 19 ms
omnisql> select count(*) from nyc_trees_2015_683k;
EXPR$0
235762432
1 rows returned.
Execution time: 17 ms, Total time: 18 ms
omnisql> select count(*) from nyc_trees_2015_683k;
EXPR$0
235762432
1 rows returned.
Execution time: 18 ms, Total time: 18 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB    26959.74 MB    28672.00 MB     1712.26 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB     2215.21 MB     4096.00 MB     1880.79 MB

omnisql> 

重启后再测试：
omnisky@omnisky-Super-Server:~/gpudb$ ./bin/omnisql -p HyperInteractive
User admin connected to database omnisci
omnisql> \l
Database | Owner
omnisci | admin
omnisql> \t
omnisci_states
omnisci_counties
omnisci_countries
flights_2008_10k
nyc_trees_2015_683k
omnisql> \gpu
omnisql> \timing
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB        0.00 MB        0.00 MB        0.00 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB        0.00 MB        0.00 MB        0.00 MB

omnisql> select count(*) from nyc_trees_2015_683k;
EXPR$0
235762432
1 rows returned.
Execution time: 772 ms, Total time: 773 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB        0.00 MB        0.00 MB        0.00 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB        0.00 MB     4096.00 MB     4096.00 MB

omnisql> select count(*) from nyc_trees_2015_683k;
EXPR$0
235762432
1 rows returned.
Execution time: 297 ms, Total time: 298 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB        0.00 MB        0.00 MB        0.00 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB        0.00 MB     4096.00 MB     4096.00 MB

omnisql> select count(*) from nyc_trees_2015_683k;
EXPR$0
235762432
1 rows returned.
Execution time: 25 ms, Total time: 26 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB        0.00 MB        0.00 MB        0.00 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB        0.00 MB     4096.00 MB     4096.00 MB

omnisql> select count(*) from nyc_trees_2015_683k where tree_id='2902';
EXPR$0
333
1 rows returned.
Execution time: 1180 ms, Total time: 1181 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB     2215.21 MB     4096.00 MB     1880.79 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB     2215.21 MB     4096.00 MB     1880.79 MB

omnisql> select count(*) from nyc_trees_2015_683k where tree_id='2902';
EXPR$0
333
1 rows returned.
Execution time: 254 ms, Total time: 256 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB     2215.21 MB     4096.00 MB     1880.79 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB     2215.21 MB     4096.00 MB     1880.79 MB

omnisql> select count(*) from nyc_trees_2015_683k where tree_id='2902';
EXPR$0
333
1 rows returned.
Execution time: 38 ms, Total time: 40 ms
omnisql> select count(*) from nyc_trees_2015_683k where tree_id='2900';
EXPR$0
356
1 rows returned.
Execution time: 30 ms, Total time: 31 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB     2215.21 MB     4096.00 MB     1880.79 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB     2215.21 MB     4096.00 MB     1880.79 MB

omnisql> select count(*) from nyc_trees_2015_683k where tree_id='1902';
EXPR$0
328
1 rows returned.
Execution time: 32 ms, Total time: 32 ms
omnisql> \memory_summary
OmniSci Server CPU Memory Summary:
            MAX            USE      ALLOCATED           FREE
   206260.19 MB     2215.21 MB     4096.00 MB     1880.79 MB

OmniSci Server GPU Memory Summary:
[GPU]            MAX            USE      ALLOCATED           FREE
  [0]    10634.25 MB     2215.21 MB     4096.00 MB     1880.79 MB

omnisql> select count(*) from nyc_trees_2015_683k where tree_id='1901';
EXPR$0
0
1 rows returned.
Execution time: 38 ms, Total time: 39 ms
omnisql> select count(*) from nyc_trees_2015_683k where tree_id='1901';
EXPR$0
0
1 rows returned.
Execution time: 30 ms, Total time: 31 ms
omnisql> select count(*) from nyc_trees_2015_683k where tree_id='1901';
EXPR$0
0
1 rows returned.
Execution time: 28 ms, Total time: 29 ms
omnisql> select count(*) from nyc_trees_2015_683k where tree_id='1901';
EXPR$0
0
1 rows returned.
Execution time: 28 ms, Total time: 29 ms
omnisql>

关于heavydb的下载：
https://releases.heavy.ai/os/tar/
image.png
安装：
https://docs.heavy.ai/installation-and-configuration/installation/installing-on-ubuntu/centos-yum-gpu-ee

墨力计划gpu数据库heavydb
最后修改时间：2023-09-21 12:19:38
「喜欢这篇文章，您的关注和赞赏是给作者最好的鼓励」
关注作者
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文章的来源（墨天轮），文章链接，文章作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

评论