作为数据仓库，greenPlum DB号称是世界上第一个开源的MPP架构的数据仓库，最初他基于postgresql，并且添加了一些大量数据库方面的创新。他可以提供PB级别的数据分析能力，支持大数据量的高性能分析查询。
之前因为工作的需要，我也曾编译过GreenPlum数据库的开源版本并且部署实施，今天回顾一下他的编译打包流程。
1. 修改系统参数
# cp /etc/sysctl.conf /etc/sysctl.bak # vi /etc/sysctl.conf kernel.shmmax = 500000000 kernel.shmmni = 4096 kernel.shmall = 4000000000 kernel.sem = 250 512000 100 2048 kernel.sysrq = 1 kernel.core_uses_pid = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.msgmni = 2048 net.ipv4.tcp_syncookies = 1 net.ipv4.ip_forward = 0 net.ipv4.conf.default.accept_source_route = 0 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_max_syn_backlog = 4096 net.ipv4.conf.all.arp_filter = 1 net.ipv4.ip_local_port_range = 1025 65535 net.core.netdev_max_backlog = 10000 net.core.rmem_max = 2097152 net.core.wmem_max = 2097152 vm.overcommit_memory = 2
|
生效
# sysctl –p
|
2. 修改/etc/security/limits.conf
# cp /etc/security/limits.conf /etc/security/limits.bak # vi /etc/security/limits.conf * soft nofile 65536 * hard nofile 65536 * soft nproc 131072 * hard nproc 131072
|
centos 6x系列还需要修改如下参数
# cp /etc/security/limits.d/90-nproc.conf /etc/security/limits.d/90-nproc.bak # vi /etc/security/limits.d/90-nproc.conf * soft nofile 65536 * hard nofile 65536 * soft nproc 131072 * hard nproc 131072
|
3. 修改磁盘调度
# echo deadline > /sys/block/sbd/queue/scheduler
|
4. 修改磁盘预读
# blockdev --setra 16384 /dev/sda* 设置成开机启动 # vi /etc/rc.local blockdev --setra 16384 /dev/sda*
|
5. 关闭防火墙
# service iptables stop # setenforce 0
|
6. 修改语言环境
# echo 'LANG="en_US.UTF-8"' > /etc/sysconfig/i18n
|
7. 配置好yum源，安装好所需的要依赖包，并且检查服务器时间
在本次编译安装过程中，我使用的是163的yum源。
# wget -O /etc/yum.repos.d/CentOS6-Base-163.repo http://mirrors.163.com/.help/CentOS6-Base-163.repo
# yum -y install rsync coreutils glib2 lrzsz sysstat e4fsprogs xfsprogs ntp curl-devel bzip2-deve readline-devel zlib zlib-devel openssl openssl-devel pam pam-devel libxml2-devel libxslt-devel python-devel tcl-devel gcc make smartmontools flex bison perl perl-devel perl-ExtUtils* OpenIPMI-tools openldap openldap-devel logrotate python-py gcc-c++ apr-devel libevent-devel bzip2-devel
# ntpdate s1a.time.edu.cn |
8. 安装python库
# wget http://mirrors.sohu.com/python/2.7.12/Python-2.7.12.tgz # tar xvf Python-2.7.12.tar.xz # cd Python-2.7.12 # ./configure --prefix=/tmp/ext/python # make && make install
# wget https://bootstrap.pypa.io/get-pip.py # export PATH=$PATH:/tmp/ext/python/bin/ # python2.7 get-pip.py # pip install paramiko # pip install psi # pip install lockfile # pip install epydoc # pip install setuptools # pip install psutil
|
9. 编译gpdb
# ./configure --prefix=/gpdb \ --with-gssapi --with-pgport=5432 \ --with-libedit-preferred --with-perl \ --with-python --with-openssl --with-pam \ --with-krb5 --with-ldap --with-libxml # make && make install
|
10. 拷贝lib库到lib目录下
# cp /usr/lib64/libpython2.6.so.1.0 /gpdb/lib/ # cp -r /tmp/ext /gpdb/ext
|
11. 修改环境变量：
# vi greenplum_path.sh # GPHOME="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
12. 将创建好的文件打包
# tar -zcvf gpdb.tar.gz /gpdb
|
至此，green plum已经打包完成，可以在服务器节点上初始化集群了。
