Hive Impala和Hue集成LDAP

数据湖 2020-09-22

1201

生产环境中Hive Impala Hue等组件经常需要集成openldap，方便管理用户，本文主要介绍这三个常用组件的集成。之前写过一篇文章，为Ranger集成OpenLDAP认证，后续会有更多的组件需要集成openldap。

配置Hive集成ldap

添加相关配置，保存并重启

用ldap用户hive登录，可以登录成功

[root@cdh1 ~]# beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.3-1.cdh7.1.3.p0.4992530/jars/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.3-1.cdh7.1.3.p0.4992530/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.3-1.cdh7.1.3.p0.4992530/jars/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.3-1.cdh7.1.3.p0.4992530/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://cdh1.macro.com:2181,cdh2.macro.com:2181,cdh3.macro.com:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Enter username for jdbc:hive2://cdh1.macro.com:2181,cdh2.macro.com:2181,cdh3.macro.com:2181/default: hive
Enter password for jdbc:hive2://cdh1.macro.com:2181,cdh2.macro.com:2181,cdh3.macro.com:2181/default: ******
20/09/22 12:11:36 [main-EventThread]: ERROR imps.EnsembleTracker: Invalid config event received: {server.1=cdh3.macro.com:3181:4181:participant, version=0, server.3=cdh1.macro.com:3181:4181:participant, server.2=cdh2.macro.com:3181:4181:participant}
20/09/22 12:11:36 [main-EventThread]: ERROR imps.EnsembleTracker: Invalid config event received: {server.1=cdh3.macro.com:3181:4181:participant, version=0, server.3=cdh1.macro.com:3181:4181:participant, server.2=cdh2.macro.com:3181:4181:participant}
20/09/22 12:11:36 [main]: INFO jdbc.HiveConnection: Connected to cdh1.macro.com:10000
Connected to: Apache Hive (version 3.1.3000.7.1.3.0-100)
Driver: Hive JDBC (version 3.1.3000.7.1.3.0-100)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.1.3000.7.1.3.0-100 by Apache Hive
0: jdbc:hive2://cdh1.macro.com:2181,cdh2.macr>

配置Impala集成ldap

进入Impala配置界面，添加相关配置

使用ldap认证登录impala-shell，错误密码登录报错

[root@cdh2 yum.repos.d]# impala-shell -l -u hive --auth_creds_ok_in_clear
Starting Impala Shell using LDAP-based authentication
LDAP password for hive: 
Error connecting: TTransportException, TSocket read 0 bytes
***********************************************************************************
Welcome to the Impala shell.
(Impala Shell v3.4.0-SNAPSHOT (2540278) built on Wed Aug  5 11:07:32 UTC 2020)


After running a query, type SUMMARY to see a summary of where time was spent.
***********************************************************************************


LDAP authentication is enabled, but the connection to Impala is not secured by TLS.
ALL PASSWORDS WILL BE SENT IN THE CLEAR TO IMPALA.
[Not connected] >

正确密码登录成功，且可使用

[root@cdh2 yum.repos.d]# impala-shell -l -u hive --auth_creds_ok_in_clear
Starting Impala Shell using LDAP-based authentication
LDAP password for hive: 
Opened TCP connection to cdh2.macro.com:21000
Connected to cdh2.macro.com:21000
Server version: impalad version 3.4.0-SNAPSHOT RELEASE (build 25402784335c39cc24076d71dab7a3ccbd562094)
***********************************************************************************
Welcome to the Impala shell.
(Impala Shell v3.4.0-SNAPSHOT (2540278) built on Wed Aug  5 11:07:32 UTC 2020)


To see a summary of a query's progress that updates in real-time, run 'set
LIVE_PROGRESS=1;'.
***********************************************************************************


LDAP authentication is enabled, but the connection to Impala is not secured by TLS.
ALL PASSWORDS WILL BE SENT IN THE CLEAR TO IMPALA.
[cdh2.macro.com:21000] default> show tables;
Query: show tables
+------------------------------+
| name                         |
+------------------------------+
| fire                         |
| kylin_account                |
| kylin_cal_dt                 |
| kylin_category_groupings     |
| kylin_country                |
| kylin_sales                  |
| managed_us_delay_flights_tbl |
| people                       |
| sum_tmp                      |
| test                         |
| test_table                   |
| us_delay_flights_tbl         |
+------------------------------+
Fetched 12 row(s) in 0.44s