Confluent Platform：企业级重磅Kafka服务平台实操

大数据从业者 2021-08-27

2009

概述

Confluent Platform是一个全面的数据流平台，使您能够轻松地访问、存储和管理连续实时数据流。Confluent由Apache Kafka®的原始创建者创建，通过企业级功能扩展了Kafka的优势，同时消除了Kafka管理或监控的负担。如今，《财富》100强中80%以上的企业都采用了数据流技术，其中大部分都采用了Confluent技术。Confluent Platform的每个版本都包括最新版本的Kafka以及其他工具和服务，这些工具和服务使构建和管理事件流平台变得更加容易。ConfluentPlatform提供社区和商业许可功能，补充和增强您的kafka部署。

企业版特性详细介绍:

https://docs.confluent.io/platform/current/platform.html#overview-of-cp-s-enterprise-features

社区版特性详细介绍:

https://docs.confluent.io/platform/current/platform.html#community-features

下载、安装、启动

[root@felixzhbigdata]# curl -O http://packages.confluent.io/archive/6.2/confluent-6.2.0.tar.gz

[root@felixzh bigdata]# tar -zxvfconfluent-6.2.0.tar.gz

[root@felixzh confluent-6.2.0]# exportCONFLUENT_HOME=/home/bigdata/confluent-6.2.0
[root@felixzh confluent-6.2.0]#bin/confluent local services start

创建测试Topic

[root@felixzh confluent-6.2.0]#bin/kafka-topics --create \
>  --bootstrap-server localhost:9092  \
>  --replication-factor 1 \
>  --partitions 1 \
>  --topic pageviews

安装Kafka connector、生成测试数据

[root@felixzh confluent-6.2.0]# curl -L -O-H 'Accept: application/vnd.github.v3.raw' \
https://api.github.com/repos/confluentinc/kafka-connect-datagen/contents/config/connector_pageviews_cos.config


[root@felixzh confluent-6.2.0]# catconnector_pageviews_cos.config

[root@felixzh confluent-6.2.0]# curl -XPOST -H "Content-Type: application/json" \
>  --data @connector_pageviews_cos.config http://localhost:8083/connectors

注意：遇到上述错误，需要提前安装kafka-connect-datagen

[root@felixzh confluent-6.2.0]# bin/confluent-hubinstall confluentinc/kafka-connect-datagen:0.5.0
The component can be installed in any ofthe following Confluent Platform installations:
  1./home/bigdata/confluent-6.2.0 (based on $CONFLUENT_HOME)
  2./home/bigdata/confluent-6.2.0 (found in the current directory)
  3./home/bigdata/confluent-6.2.0 (where this tool is installed)
Choose one of these to continue theinstallation (1-3): 1
Do you want to install this into/home/bigdata/confluent-6.2.0/share/confluent-hub-components? (yN) y
 
 
Component's license:
Apache License 2.0
https://www.apache.org/licenses/LICENSE-2.0
I agree to the software license agreement(yN) y
 
Downloading component Kafka Connect Datagen0.5.0, provided by Confluent, Inc. from Confluent Hub and installing into/home/bigdata/confluent-6.2.0/share/confluent-hub-components
Detected Worker's configs:
  1.Standard: home/bigdata/confluent-6.2.0/etc/kafka/connect-distributed.properties
  2.Standard: home/bigdata/confluent-6.2.0/etc/kafka/connect-standalone.properties
  3.Standard:/home/bigdata/confluent-6.2.0/etc/schema-registry/connect-avro-distributed.properties
  4.Standard: home/bigdata/confluent-6.2.0/etc/schema-registry/connect-avro-standalone.properties
  5.Based on CONFLUENT_CURRENT: /tmp/confluent.544161/connect/connect.properties
  6.Used by Connect process with PID 24953:/tmp/confluent.544161/connect/connect.properties
Do you want to update all detected configs?(yN) y
 
Adding installation directory to pluginpath in the following files:
 /home/bigdata/confluent-6.2.0/etc/kafka/connect-distributed.properties
 /home/bigdata/confluent-6.2.0/etc/kafka/connect-standalone.properties
  /home/bigdata/confluent-6.2.0/etc/schema-registry/connect-avro-distributed.properties
 /home/bigdata/confluent-6.2.0/etc/schema-registry/connect-avro-standalone.properties
 /tmp/confluent.544161/connect/connect.properties
 /tmp/confluent.544161/connect/connect.properties
 
Completed

重启启动confluent platform

[root@felixzh confluent-6.2.0]#bin/confluent local services stop
[root@felixzh confluent-6.2.0]#bin/confluent local services start

再次提交connect任务

Confluent Control Center页面

http://felixzh:9021/

http://felixzh:9021/clusters

http://felixzh:9021/clusters/tXNNJiVWTOSmLvkGlFKItw/overview

大数据

文章转载自大数据从业者，如果涉嫌侵权，请发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

Confluent Platform：企业级重磅Kafka服务平台实操

评论