Impala负载均衡实现—Nginx

更多精彩，请点击上方蓝字关注我们！

1 演示环境

集群未启用Kerberos
Nginx1.12.2
CM和CDH版本5.13.0
用root用户操作

2 操作演示

Nginx服务安装及启停

• 下载Nginx安装包，下载地址：http://nginx.org/download/nginx-1.12.2.tar.gz

• 选择集群中任意一台服务器用来安装Nginx服务或者选用一台独立的服务器用来部署Nginx，需要确保Nginx所在的服务器与集群中所有的Impala节点网络是通的

• 包括端口号21050

• 解压nginx-1.21.2.tar.gz压缩包，并进行编译安装

• 注意：必须编译stream模块

• 默认Nginx的安装目录为/usr/local/nginx

[root@ip-172-31-6-148 ~]# tar -zxf nginx-1.12.2.tar.gz  
[root@ip-172-31-6-148 ~]# cd nginx-1.12.2
[root@ip-172-31-6-148 nginx-1.12.2]# ./configure --with-stream
[root@ip-172-31-6-148 nginx-1.12.2]# make && make install

• 将nginx服务加如系统自启动服务，在/etc/init.d目录下创建nginx脚本，内容如下

• Nginx的home目录，根据自己安装的目录进行调整。

#!/bin/sh
#chkconfig: - 85 15
#description: Http server daemon

DESC="nginx daemon"
NAME=nginx
DAEMON=/usr/local/nginx/sbin/nginx

set -e
[ -x "$DAEMON" ] || exit 0

do_start() {
$DAEMON || echo -n "nginx is already running";
}

do_stop() {
$DAEMON -s stop || echo -n "nginx stop failed"
}

do_reload() {
$DAEMON -s reload || echo -n "nginx reload failed"
}

case "$1" in
start)
do_start
;;
stop)
do_stop
;;
reload|graceful)
do_reload
;;
restart)
do_stop
sleep 2
do_start
;;
*)
echo "Usage: $SCRIPTNAME {start|stop|reload|restart}" >&2
exit 3
;;
esac

exit 0

• 设置自启动脚本执行权限

[root@ip-172-31-6-148 ~]# chmod +x etc/init.d/nginx 
[root@ip-172-31-6-148 ~]# ll etc/init.d/nginx 
-rwxr-xr-x 1 root root 559 Nov  8 15:52 etc/init.d/nginx
[root@ip-172-31-6-148 ~]#

• 将nginx服务加入系统自启动并设置开机启动

[root@ip-172-31-6-148 ~]# chkconfig --add nginx
[root@ip-172-31-6-148 ~]# chkconfig nginx on

• 启动与停止Nginx

[root@ip-172-31-5-190 nginx-1.12.2]# service nginx start
[root@ip-172-31-5-190 nginx-1.12.2]# netstat -an |grep 80

[root@ip-172-31-5-190 nginx-1.12.2]# service nginx stop

• 测试Nginx是否正常访问，在浏览器输入http://hostname
  

  
  

配置Impala负载均衡策略

• 修改/usr/local/nginx/conf/nginx.conf文件，在文件末尾增加如下配置

stream{
  log_format basic '$remote_addr [$time_local] ' '$protocol $status $bytes_sent $bytes_received' '$session_time';

  upstream impala { #impala daemon
    least_conn; #路由策略:least_conn:最少连接
    server ip-172-31-9-33.fayson.com:21000;
    server ip-172-31-5-190.fayson.com:21000;
    server ip-172-31-10-118.fayson.com:21000;
  }
  upstream impala-jdbc { 
    least_conn;
    server ip-172-31-9-33.fayson.com:21050;
    server ip-172-31-5-190.fayson.com:21050;
    server ip-172-31-10-118.fayson.com:21050;
  }
  server{ #impala 负载均衡
    listen 21001;
    proxy_pass impala;
  }
  server{ #impala jdbc 负载均衡
    listen 21051;
    proxy_pass impala-jdbc;
  }
}

• 重启Nginx服务

[root@ip-172-31-5-190 ~]# service nginx restart

测试Impala负载均衡

• 在ip-172-31-10-118.fayson.com节点使用impala-shell连接Impala

[root@ip-172-31-6-148 ~]# impala-shell -i ip-172-31-5-190.fayson.com:21001
[ip-172-31-5-190.fayson.com:21001] > select * from my_first_table;
Query submitted at: 2017-11-08 16:56:51 (Coordinator: http://ip-172-31-10-118.fayson.com:25000)
+----+------+
| id | name |
+----+------+
| 1  | john |
| 2  | tom  |
| 3  | jim  |
+----+------+
Fetched 3 row(s) in 1.57s
[ip-172-31-5-190.fayson.com:21001] >

• 同时在ip-172-31-5-190.fayson.com节点使用impala-shell连接Impala

[root@ip-172-31-5-190 nginx-1.12.2]# impala-shell -i ip-172-31-5-190.fayson.com:21001
...
[ip-172-31-5-190.fayson.com:21001] > select * from my_first_table;
Query: select * from my_first_table
Query submitted at: 2017-11-08 16:59:53 (Coordinator: http://ip-172-31-5-190.fayson.com:25000)
+----+------+
| id | name |
+----+------+
| 2  | tom  |
| 3  | jim  |
| 1  | john |
+----+------+
Fetched 3 row(s) in 1.40s
[ip-172-31-5-190.fayson.com:21001] >

• 两个终端测试执行sql操作时，连接的不为同一个Impala Daemon

Impala JDBC连接测试

• 创建测试工程impalajdbc

• 将Impala JDBC驱动包加入工程的lib目录下

• 将jar包加入环境classpath

• 示例代码

package com.cloudera;


import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;

public class App {
    static String JDBC_DRIVER = "com.cloudera.impala.jdbc41.Driver";
    static String CONNECTION_URL = "jdbc:impala://54.254.149.113:21051";

    public static void main( String[] args ) {
        Connection con = null;
        ResultSet rs = null;
        PreparedStatement ps = null;
        try {
            Class.forName(JDBC_DRIVER);
            con = DriverManager.getConnection(CONNECTION_URL);
            //ps = con.prepareStatement("show tables;");
            ps = con.prepareStatement("select * from my_first_table;");
            rs = ps.executeQuery();
            while (rs.next()) {
                System.out.println("s1="+rs.getString(1) + ",  s2="+rs.getString(2));
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            //关闭 rs、ps 和 con
        }
    }
}

常见问题

• Nginx编译时报错./configure:error: C compiler cc is not found

[root@ip-172-31-6-148 nginx-1.12.2]# ./configure 
checking for OS
 + Linux 2.6.32-431.el6.x86_64 x86_64
checking for C compiler ... not found
./configure: error: C compiler cc is not found
[root@ip-172-31-6-148 nginx-1.12.2]#

• 解决方法：

yum -y install gcc gcc-c++

• Nginx编译报错./configure:error: the HTTP rewrite module requires the PCRE library

./configure: error: the HTTP rewrite module requires the PCRE library.
You can either disable the module by using --without-http_rewrite_module
option, or install the PCRE library into the system, or build the PCRE library
statically from the source with nginx by using --with-pcre=option.

• 解决方法：

yum -y install pcre-devel

关注公众号：领取精彩视频课程&海量免费语音课程