0188.S 使用Docker部署StarRocks V2.4.0测试环境

如何使用docker构建StarRocks 2.4.0测试版本？也可直接拉取我创建的docker image，完成快速测试环境构建。

CentOS 7.6;StarRocks-2.4.0-test-1104.tar.gz;java-1.8.0-openjdk-1.8.0.352.b08-2.el7_9.x86_64;docker 20.10.12。

单台主机运行FE、BE，后续将通过Kubernetes构建StarRocks ComputeNode组件。

# 创建目录

mkdir -p /data/deploy/ && cd /data/deploy/

# 创建Dockerfile

cat >> Dockerfile <<-'EOF'
FROM centos:centos7
# Prepare StarRocks Installer.
RUN yum -y install wget
RUN mkdir -p data/deploy/
# RUN wget -SO data/deploy/StarRocks-x.x.x.tar.gz <url_to_download_specific_ver_of_starrocks>
RUN wget -SO data/deploy/StarRocks-2.4.0-test-1104.tar.gz "http://cdn-release.starrocks.com/StarRocks-2.4.0-test-1104.tar.gz?Expires=1668996477&OSSAccessKeyId=LTAI4GFYjbX9e7QmFnAAvkt8&Signature=zUNVpBk3oqb4MZTcTnflyiI5XKo%3D"
RUN cd data/deploy/ && tar zxf StarRocks-2.4.0-test-1104.tar.gz && rm -rf StarRocks-2.4.0-test-1104.tar.gz 
# Install Java JDK.
RUN yum -y install java-1.8.0-openjdk-devel.x86_64
RUN rpm -ql java-1.8.0-openjdk-devel.x86_64 | grep bin$
#RUN usr/lib/jvm/java-1.8.0-openjdk-1.8.0.342.b07-1.el7_9.x86_64/bin/java -version
RUN /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.352.b08-2.el7_9.x86_64/bin/java -version
# Create directory for FE meta and BE storage in StarRocks.
RUN mkdir -p data/deploy/StarRocks-2.4.0-test-1104/fe/meta
RUN jps
RUN mkdir -p data/deploy/StarRocks-2.4.0-test-1104/be/storage
# Install relevant tools.
RUN yum -y install mysql net-tools telnet
# Run Setup script.
COPY run_script.sh data/deploy/run_script.sh
RUN chmod +x data/deploy/run_script.sh
CMD /data/deploy/run_script.sh
EOF

注意：

将以上 <url_to_download_specific_ver_of_starrocks> 替换为实际下载地址;

并将 StarRocks-x.x.x 替换为实际安装版本。

构建脚本文件run_script.sh以配置并启动 StarRocks。

cat >> data/deploy/run_script.sh <<-'EOF'
#!/bin/bash

# Set JAVA_HOME.
#export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.342.b07-1.el7_9.x86_64
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.352.b08-2.el7_9.x86_64

# Set StarRocks_DIR
export SR_DIR=StarRocks-2.4.0-test-1104

# Start FE.
cd data/deploy/$SR_DIR/fe/bin/
./start_fe.sh --daemon

# Start BE.
cd data/deploy/$SR_DIR/be/bin/
./start_be.sh --daemon

# Sleep until the cluster starts.
sleep 30;
# Set BE server IP.
# IP=$(ifconfig eth0 | grep 'inet' | cut -d: -f2 | awk '{print $2}')
IP=$(hostname -I | awk '{print $1}')
mysql -uroot -h${IP} -P 9030 -e "alter system add backend '${IP}:9050';"

# Loop to detect the process.
while sleep 60; do
  ps aux | grep starrocks | grep -q -v grep
  PROCESS_STATUS=$?
  if [ $PROCESS_STATUS -ne 0 ]; then
    echo "one of the starrocks process already exit."
    exit 1;
  fi
done
EOF

注意：将以上 StarRocks-x.x.x 替换为实际安装版本。

运行以下命令搭建Docker镜像

[root@rundba deploy]# docker build --no-cache --progress=plain -t starrocks-test:2.4.0 .
Sending build context to Docker daemon  4.608kB
Step 1/15 : FROM centos:centos7
centos7: Pulling from library/centos
2d473b07cdd5: Downloading 
...
Complete!
Removing intermediate container 6ce41a1b016c
 ---> 3188052a1380
Step 13/15 : COPY run_script.sh data/deploy/run_script.sh
 ---> ad28d617841a
Step 14/15 : RUN chmod +x data/deploy/run_script.sh
 ---> Running in 9e333d78bff9
Removing intermediate container 9e333d78bff9
 ---> c0139e4675d7
Step 15/15 : CMD data/deploy/run_script.sh
 ---> Running in 3f50b1eeb713
Removing intermediate container 3f50b1eeb713
 ---> 26cf814018e6
Successfully built 26cf814018e6
Successfully tagged starrocks-test:2.4.0

查看Docker镜像:

[root@rundba deploy]# docker images
REPOSITORY       TAG       IMAGE ID       CREATED          SIZE
starrocks-test   2.4.0     26cf814018e6   19 seconds ago   5.2GB
centos           centos7   eeb6ee3f44bd   13 months ago    204MB

运行以下命令启动Docker容器

[root@rundba ~]# docker run -p 9030:9030 -p 8030:8030 -p 8040:8040 --privileged=true -itd --name starrocks-test starrocks-test:2.4.0
4acf39e8b961b9d6f1876da9848059c40c2be85730aeef385dbeaf263c93cd83

StarRocks运行成功:

[root@rundba ~]# docker ps -a
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS                                                                                                                             NAMES
4acf39e8b961   starrocks-test:2.4.0   "/bin/sh -c data/de…"   17 seconds ago   Up 16 seconds   0.0.0.0:8030->8030/tcp, :::8030->8030/tcp, 0.0.0.0:8040->8040/tcp, :::8040->8040/tcp, 0.0.0.0:9030->9030/tcp, :::9030->9030/tcp   starrocks-test

登录容器:

[root@rundba ~]# docker exec -it 4acf39e8b961 bin/bash
[root@4acf39e8b961 ]#

在容器中连接StarRocks:

当 Docker 容器成功启动后，运行以下命令连接 StarRocks。

[root@4acf39e8b961 ]# mysql -uroot -h127.0.0.1 -P 9030Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.1.0 StarRocks version 2.4.0-TEST-1104

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]>

您可以运行以下 SQL 确认 StarRocks 是否部署成功。

CREATE DATABASE TEST;

USE TEST;

CREATE TABLE `sr_on_mac` (
 `c0` int(11) NULL COMMENT "",
 `c1` date NULL COMMENT "",
 `c2` datetime NULL COMMENT "",
 `c3` varchar(65533) NULL COMMENT ""
) ENGINE=OLAP 
DUPLICATE KEY(`c0`)
PARTITION BY RANGE (c1) (
  START ("2022-02-01") END ("2022-02-10") EVERY (INTERVAL 1 DAY)
)
DISTRIBUTED BY HASH(`c0`) BUCKETS 1 
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"storage_format" = "DEFAULT"
);

insert into sr_on_mac values (1, '2022-02-01', '2022-02-01 10:47:57', '111');
insert into sr_on_mac values (2, '2022-02-02', '2022-02-02 10:47:57', '222');
insert into sr_on_mac values (3, '2022-02-03', '2022-02-03 10:47:57', '333');

MySQL [TEST]> select * from sr_on_mac where c1 >= '2022-02-02';
+------+------------+---------------------+------+
| c0   | c1         | c2                  | c3   |
+------+------------+---------------------+------+
|    2 | 2022-02-02 | 2022-02-02 10:47:57 | 222  |
|    3 | 2022-02-03 | 2022-02-03 10:47:57 | 333  |
+------+------------+---------------------+------+
2 rows in set (0.14 sec)

如果无错误返回，则表明您已成功在 Docker环境中部署StarRocks。

1) 查看FE

MySQL [TEST]> SHOW PROC '/frontends';
+-------------------------------+------------+-------------+----------+-----------+---------+--------+-----------+------+-------+-------------------+---------------------+----------+--------+---------------------+-------------------------+
| Name                          | IP         | EditLogPort | HttpPort | QueryPort | RpcPort | Role   | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat       | IsHelper | ErrMsg | StartTime           | Version                 |
+-------------------------------+------------+-------------+----------+-----------+---------+--------+-----------+------+-------+-------------------+---------------------+----------+--------+---------------------+-------------------------+
| 172.17.0.2_9010_1667799234791 | 172.17.0.2 | 9010        | 8030     | 9030      | 9020    | LEADER | 464381294 | true | true  | 97                | 2022-11-07 05:38:03 | true     |        | 2022-11-07 05:34:04 | 2.4.0-TEST-1104-4eaf688 |
+-------------------------------+------------+-------------+----------+-----------+---------+--------+-----------+------+-------+-------------------+---------------------+----------+--------+---------------------+-------------------------+
1 row in set (0.03 sec)

2) 查看BE

MySQL [TEST]> SHOW PROC '/backends';
+-----------+------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------+-------------------------+--------------------------------------------------------+-------------------+-------------+----------+
| BackendId | IP         | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime       | LastHeartbeat       | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | ErrMsg | Version                 | Status                                                 | DataTotalCapacity | DataUsedPct | CpuCores |
+-----------+------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------+-------------------------+--------------------------------------------------------+-------------------+-------------+----------+
| 10002     | 172.17.0.2 | 9050          | 9060   | 8040     | 8060     | 2022-11-07 05:34:27 | 2022-11-07 05:38:08 | true  | false                | false                 | 39        | 2.595 KB         | 282.763 GB    | 291.365 GB    | 2.95 %  | 2.95 %         |        | 2.4.0-TEST-1104-4eaf688 | {"lastSuccessReportTabletsTime":"2022-11-07 05:37:28"} | 282.763 GB        | 0.00 %      | 16       |
+-----------+------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------+-------------------------+--------------------------------------------------------+-------------------+-------------+----------+
1 row in set (0.01 sec)

3) 通过浏览器查看FE、BE、CN信息

通过浏览器查看FE信息：

http://HOST_IP:8030/

其中HOST_IP为docker宿主机IP。

通过浏览器查看BE信息：

http://HOST_IP:8040/

其中HOST_IP为docker宿主机IP。

通过浏览器查看CN信息：

FE主页 -> 点击“system” -> compute_nodes，即可查看，效果同show compute nodes。

目前我已经将制作好的StarRocks-2.4.0-test-1104镜像上传到Hub Docker，可供其它爱好者进行个人测试使用，直接下载镜像并运行，较快运行StarRocks FE、BE测试环境，可免去上述操作步骤。

从Hub docker拉取镜像：

docker pull landnow/starrocks-test:v2.4.0

运行StarRocks：

docker run -p 9030:9030 -p 8030:8030 -p 8040:8040 --privileged=true -itd --name starrocks-test landnow/starrocks-test:v2.4.0

https://docs.starrocks.io/zh-cn/latest/administration/deploy_with_docker