DuckDB~01–安装

##

DuckDB~01–安装

1、概述

The DuckDB team is *very happy* to announce that today we’re releasing DuckDB version 1.0.0, codename “Snow Duck” (anas nivis).

DuckDB发布了1.0.0的版本，非常火，看了下同行，已经有单子了，所以学习搞起来。

2、binary方法安装

2.1、关闭防火墙和selinux

sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
setenforce 0

systemctl stop firewalld.service && systemctl disable firewalld.service && systemctl status firewalld.service

2.2、创建目录

mkdir -p /duckdb/{app,data,archive,backup,log,wal}

2.3、上传安装包

下载地址：

https://github.com/duckdb/duckdb/releases/download/v1.0.0/duckdb_cli-linux-amd64.zip

上传到/duckdb/app

2.4、安装一些依赖包

dnf -y install git 
dnf -y install g++
dnf -y install cmake
dnf -y install python3-devel
dnf -y install https://dl.rockylinux.org/pub/rocky/9/CRB/x86_64/os/Packages/n/ninja-build-1.10.2-6.el9.x86_64.rpm
dnf -y install openssl-devel
dnf -y install https://dl.rockylinux.org/pub/rocky/9/CRB/x86_64/os/Packages/u/unixODBC-devel-2.3.9-4.el9.x86_64.rpm
python3 -m pip install --upgrade pip

2.5、配置环境变量

[root@mydb ~]# cat ~/.bash_profile
export LANG=en_US.utf8
export PATH=/duckdb/app:$PATH:.

2.6、解压运行

unzip duckdb_cli-linux-amd64.zip
./duckdb

2.7、基本测试

CREATE TABLE people (id INTEGER, name VARCHAR);
INSERT INTO people VALUES (1, 'Mark'), (2, 'Hannes');
SELECT * FROM people;

2.8、其他操作

两次Ctrl+c是退出或者ctrl+d，或者.exit

3、编译安装

3.1、基础步骤

与1.1/1.2/1.4/1.5相同

3.2、下载安装包

这里使用git,或者手动下载源码上传到/duckdb/app

[root@mydb ~]# cd /duckdb/app
[root@mydb app]# git clone https://github.com/duckdb/duckdb.git
[root@mydb app]# cd duckdb/

#有可能网络问题导致git失败，再次git就行

3.3、编译安装

编译安装，只需要make就可以，如果需要默认的make选项，那么使用

GEN=ninja make -j 4

推荐编译其他选项

GEN=ninja BUILD_PYTHON=1 BUILD_SHELL=1 BUILD_BENCHMARK=1 BUILD_JDBC=1 BUILD_ODBC=1 BUILD_HTTPFS=1 BUILD_AUTOCOMPLETE=1 BUILD_ICU=1 BUILD_TPCH=1 BUILD_TPCDS=1 BUILD_TPCE=1 BUILD_FTS=1 BUILD_JEMALLOC=1 BUILD_JSON=1 BUILD_INET=1 BUILD_SQLSMITH=1 make -j 4

安装成功之后的信息

[859/859] package
Processing /duckdb/app/duckdb/tools/pythonpkg
 Installing build dependencies: started
 Installing build dependencies: still running...
 Installing build dependencies: finished with status 'done'
 Getting requirements to build wheel: started
 Getting requirements to build wheel: finished with status 'done'
 Preparing metadata (pyproject.toml): started
 Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: duckdb
 Building wheel for duckdb (pyproject.toml): started
 Building wheel for duckdb (pyproject.toml): still running...
 Building wheel for duckdb (pyproject.toml): finished with status 'done'
 Created wheel for duckdb: filename=duckdb-1.0.1.dev47-cp39-cp39-linux_x86_64.whl size=63433394 sha256=389175ffeea7a9f03ff5cfe0ab51cc3e6831f39f128d66cc9be04d172bd5cfaf
 Stored in directory: /tmp/pip-ephem-wheel-cache-1khzojk2/wheels/99/7a/3b/89a6f050dce1a012adc22906541ce67632c6e4b6d0cfd39e36
Successfully built duckdb

DEPRECATION: pyodbc 4.0.0-unsupported has a non-standard version number. pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pyodbc or contact the author to suggest that they release a version with a conforming version number. Discussion can be found at https://github.com/pypa/pip/issues/12063

Installing collected packages: duckdb

Successfully installed duckdb-1.0.1.dev47

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

3.4、验证

安装完成之后的目录是build/release/，所以需要调整下

export PATH=/duckdb/app/duckdb/build/release:$PATH:.

登录

[root@mydb duckdb]# cd build/release/

[root@mydb release]# ./duckdb
v1.0.1-dev47 9d343bf07c
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D .exit

4、问题总结

4.1、pip未升级导致make失败

如果不升级pip，会遇到下面这个错误

FAILED: CMakeFiles/duckdb_python /duckdb/app/duckdb/build/release/CMakeFiles/duckdb_python
cd /duckdb/app/duckdb/tools/pythonpkg && cmake -E env DUCKDB_BINARY_DIR=/duckdb/app/duckdb/build/release DUCKDB_COMPILE_FLAGS=\ -O3\ -DNDEBUG\ -O3\ -DNDEBUG\ \  DUCKDB_LIBS="dl duckdb_fsst duckdb_fmt duckdb_pg_query duckdb_re2 duckdb_miniz duckdb_utf8proc duckdb_hyperloglog duckdb_fastpforlib duckdb_skiplistlib duckdb_mbedtls duckdb_yyjson Threads::Threads autocomplete_extension icu_extension tpch_extension tpcds_extension fts_extension httpfs_extension json_extension jemalloc_extension sqlsmith_extension parquet_extension" python3 -m pip install .
Processing /duckdb/app/duckdb/tools/pythonpkg
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpticc59z4
       cwd: /tmp/pip-req-build-l6irkb5s
  Complete output (15 lines):
  Traceback (most recent call last):
    File "/usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 349, in <module>
      main()
    File "/usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 331, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 117, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-i491_wr5/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-i491_wr5/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-i491_wr5/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 313, in <module>
  ModuleNotFoundError: No module named 'package_build'
  ----------------------------------------
WARNING: Discarding file:///duckdb/app/duckdb/tools/pythonpkg. Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpticc59z4 Check the logs for full command output.
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpticc59z4 Check the logs for full command output.
ninja: build stopped: subcommand failed.
make: *** [Makefile:281：release] 错误 1

4.2、tar包导致make失败

如果下载的tar.gz包。像下面这样，编译会遇到如下问题，建议git clone，猜测可能是包的问题

[root@mydb app]# tar -zxvf duckdb-1.0.0.tar.gz
[root@mydb app]# cd duckdb-1.0.0/

FAILED: CMakeFiles/duckdb_python /duckdb/app/duckdb-1.0.0/build/release/CMakeFiles/duckdb_python
cd /duckdb/app/duckdb-1.0.0/tools/pythonpkg && cmake -E env DUCKDB_BINARY_DIR=/duckdb/app/duckdb-1.0.0/build/release DUCKDB_COMPILE_FLAGS=\ -O3\ -DNDEBUG\ -O3\ -DNDEBUG\ \  DUCKDB_LIBS="dl duckdb_fsst duckdb_fmt duckdb_pg_query duckdb_re2 duckdb_miniz duckdb_utf8proc duckdb_hyperloglog duckdb_fastpforlib duckdb_skiplistlib duckdb_mbedtls duckdb_yyjson Threads::Threads autocomplete_extension icu_extension tpch_extension tpcds_extension fts_extension httpfs_extension json_extension jemalloc_extension sqlsmith_extension parquet_extension" python3 -m pip install .
Processing /duckdb/app/duckdb-1.0.0/tools/pythonpkg
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [35 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 386, in <module>
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 146, in setup
          _setup_distribution = dist = klass(attrs)
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 307, in __init__
          _Distribution.__init__(self, dist_attrs)
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 284, in __init__
          self.finalize_options()
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 659, in finalize_options
          ep(self)
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools_scm/_integration/setuptools.py", line 123, in infer_version
          _assign_version(dist, config)
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools_scm/_integration/setuptools.py", line 58, in _assign_version
          _version_missing(config)
        File "/tmp/pip-build-env-etjrgl45/overlay/lib/python3.9/site-packages/setuptools_scm/_get_version_impl.py", line 117, in _version_missing
          raise LookupError(
      LookupError: setuptools-scm was unable to detect version for /duckdb/app/duckdb-1.0.0.

      Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

      For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
ninja: build stopped: subcommand failed.
make: *** [Makefile:281: release] Error 1

5、安装总结

目前推荐使用binary方法安装，

编译安装会有一些问题，具体参考：https://duckdb.org/docs/dev/building/troubleshooting