暂无图片
暂无图片
暂无图片
暂无图片
暂无图片

debezium镜像迁移实践笔记

原创 数据库管理员陆美芳 2024-05-14
594

前言

因为生产环境已经上云,云上无法直接拉取国外的官方镜像,尝试了很多次无法pull成功,但是测试环境可以pull到,于是决定先在测试环境pull下来,然后通过docker save和docker load的方式完成生产环境的部署。然而,这个看起来很简单的过程,却还是遇到了一个权限方面的坑,具体不知道为啥,但是好在知道了怎么解决,以此文总结,防止自己下次再遇到,也分享给需要的朋友们,希望对您有帮助。

1、测试环境上备份镜像

# 先查看镜像名称和版本等信息
[root@192 sink-connect]# docker images
REPOSITORY                                                TAG                     IMAGE ID       CREATED         SIZE
debezium/connect                                          2.6                     ec43c5384d1c   28 hours ago    1.33GB
# 避免不必要的报错,先停止正在基于镜像运行的容器
docker stop debezium-connect
# 然后执行备份命令
docker save -o debezium-connect.tar debezium/connect:2.6

执行完备份命令,得到镜像文件 debezium-connect.tar

2、把镜像备份文件下载到本地 d:\daochu

下载镜像备份文件.png

3、上传镜像备份文件到目标服务器

首先:打开生产远程窗口的SFTP窗口,然后按下面命令操作

sftp> lcd d:\daochu
sftp> lls
containerd.io-1.6.9-3.1.el7.x86_64.rpm      debezium-connect.tar
docker-24.0.7.tgz     docker-ce-24.0.7-1.el7.x86_64.rpm
docker-ce-cli-24.0.7-1.el7.x86_64.rpm       docker-ce-selinux-17.03.3.ce-1.el7.noarch.rpm
docker-compose-plugin-2.6.0-3.el7.x86_64.rpm
epel-release-latest-7.noarch.rpm            etcloud.sql
full.dmp              information_authority20240325000001.sql
invoice_info_1.sql.tar                      mysql-master.tar
sftp> put debezium-connect.tar

等待上传完成

4、目标服务器上导入镜像

docker load -i debezium-connect.tar

5、查看导入结果

[root@iZ2ze53uc2vpneg2cdhweoZ sink-connect]# docker images
REPOSITORY                                           TAG          IMAGE ID       CREATED        SIZE
debezium/connect                                     2.6          ec43c5384d1c   29 hours ago   1.33GB

如上,表示导入成功

6、基于导入的镜像构建容器

1)编写docker-compose.yaml文件
version: '3.7'
services:
  debezium-connect:
    privileged: true
    container_name: debezium-connect
    user: root
    working_dir: /kafka
    image: debezium/connect:2.6
    restart: always
    ports:
      - 8083:8083
    environment:
      - BOOTSTRAP_SERVERS=kafka的ip地址:9092
      - GROUP_ID=debezium
      # connector相关信息持久化到kafka指定的topic
      - CONFIG_STORAGE_TOPIC=debezium_connect_configs_1
      # 存储连接器偏移量的Kafka主题的名称
      - OFFSET_STORAGE_TOPIC=debezium_connect_offsets_1
      # 存储连接器和任务状态的Kafka主题的名称
      - STATUS_STORAGE_TOPIC=debezium_connect_statuses_1
      - ENABLE_DEBEZIUM_KC_REST_EXTENSION=true
      - ENABLE_DEBEZIUM_SCRIPTING=true
    volumes:
      - "/usr/share/zoneinfo/Asia/Shanghai:/etc/localtime"
      - "/opt/dockerstore/debezium-connect/config/:/kafka/config"
      - "/opt/dockerstore/debezium-connect/data/:/kafka/data"
      - "/opt/dockerstore/debezium-connect/logs/:/kafka/logs"
 
2)构建容器
docker-compose up -d --build

7、查看容器运行状态

[root@iZ2ze53uc2vpneg2cdhweoZ source-connect]# docker ps
CONTAINER ID   IMAGE                                                           COMMAND                  CREATED         STATUS       PORTS                                                           NAMES
b6f78c97c538   debezium/connect:2.6                                            "/docker-entrypoint.…"   4 hours ago     Up 4 hours   8778/tcp, 0.0.0.0:8083->8083/tcp, :::8083->8083/tcp, 9092/tcp   debezium-connect

如上表示容器启动成功,运行正常,到此,镜像迁移成功完成!

8、遇到的问题及解决方案

在前面几次,导入镜像时,docker-compose.yaml采用和测试环境一样的脚本,内容如下:

version: '3.7'
services:
  debezium-connect:
    privileged: true
    container_name: debezium-connect
    image: debezium/connect:2.6
    restart: always
    ports:
      - 8083:8083
    environment:
      - BOOTSTRAP_SERVERS=kafka的ip地址:9092
      - GROUP_ID=debezium
      # connector相关信息持久化到kafka指定的topic
      - CONFIG_STORAGE_TOPIC=debezium_connect_configs_1
      # 存储连接器偏移量的Kafka主题的名称
      - OFFSET_STORAGE_TOPIC=debezium_connect_offsets_1
      # 存储连接器和任务状态的Kafka主题的名称
      - STATUS_STORAGE_TOPIC=debezium_connect_statuses_1
      - ENABLE_DEBEZIUM_KC_REST_EXTENSION=true
      - ENABLE_DEBEZIUM_SCRIPTING=true
    volumes:
      - "/usr/share/zoneinfo/Asia/Shanghai:/etc/localtime"
      - "/opt/dockerstore/debezium-connect/config/:/kafka/config"
      - "/opt/dockerstore/debezium-connect/data/:/kafka/data"
      - "/opt/dockerstore/debezium-connect/logs/:/kafka/logs"

这样构建容器就总无法启动,docker logs debezium-connect命令总看到如下日志:

cp: cannot create regular file ‘/kafka/config/connect-console-sink.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/connect-console-source.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/connect-distributed.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/connect-file-sink.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/connect-file-source.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/connect-log4j.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/connect-mirror-maker.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/connect-standalone.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/consumer.properties’: Permission denied
cp: cannot create directory ‘/kafka/config/kraft’: Permission denied
cp: cannot create regular file ‘/kafka/config/log4j.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/producer.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/server.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/tools-log4j.properties’: Permission denied
cp: cannot create regular file ‘/kafka/config/trogdor.conf’: Permission denied
cp: cannot create regular file ‘/kafka/config/zookeeper.properties’: Permission denied

经过几番折腾,最终大胆猜测,指定使用root运行容器,即把docker-compose.yaml改成了第6部分所示的脚本,即比测试环境的脚本多下面两行:

    user: root
    working_dir: /kafka

然后果然构建就成功了!

最后修改时间:2024-05-14 17:30:10
「喜欢这篇文章,您的关注和赞赏是给作者最好的鼓励」
关注作者
【版权声明】本文为墨天轮用户原创内容,转载时必须标注文章的来源(墨天轮),文章链接,文章作者等基本信息,否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容,欢迎发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

文章被以下合辑收录

评论