暂无图片
暂无图片
暂无图片
暂无图片
暂无图片
首页 / redis5 cluster模式部署

redis5 cluster模式部署

feelwow 2020-03-30
825

环境参考

  • centos7+

  • 三台机器

  • 每台机器均启动两个实例

  • redis版本均为redis-5.0.4

192.168.152.129 ----- 7000

192.168.152.129 ----- 7001

192.168.152.130 ----- 7004

192.168.152.130 ----- 7005

192.168.152.131 ----- 7002

192.168.152.132 ----- 7003

部署

  1. 下载源码包

  2. 在每台机器上分别解压，并编译

  3. 修改配置文件

  4. 启动

下载源码包

  1. wget http://download.redis.io/releases/redis-5.0.4.tar.gz

解压

  1. tar zxvf redis-5.0.4.tar.gz

安装

  1. cd redis-5.0.4

  2. make PREFIX=/usr/local/redis install

修改配置文件

  1. 以端口为7000的实例为例

  2. mkdir /usr/local/redis/config/{7000,7001} -p

  3. cp redis-5.0.4/redis.conf /usr/local/redis/config/70001

  5. cat >> /usr/local/redis/config/7000/redis.conf << EOF

  6. cluster-enabled yes

  7. cluster-config-file nodes-7000.conf

  8. cluster-node-timeout 5000

  9. appendonly yes

  10. EOF

  12. sed -i 's/6379/7000/g' /usr/local/redis/config/7000/redis.conf

  13. sed -i 's/daemonize no/daemonize yes/g' /usr/local/redis/config/7000/redis.conf

  14. sed -i 's@logfile ""@logfile "/var/log/redis/redis-7000.log"@g' /usr/local/redis/config/7000/redis.conf

启动

在六个实例准备好之后，便重启每一个redis实例，这里我们最好使用systemd来管理redis服务

  1. # cat redis@.service

  2. [Unit]

  3. Description=Redis

  4. After=network.target

  6. [Service]

  7. Type=forking

  8. PIDFile=/var/run/redis_%i.pid

  9. ExecStart=/usr/local/redis/bin/redis-server /usr/local/redis/config/%i/redis.conf

  10. ExecReload=/bin/kill -s HUP $MAINPID

  11. ExecStop=/bin/kill -s QUIT $MAINPID

  12. PrivateTmp=true

  14. [Install]

  15. WantedBy=multi-user.target

  17. systemctl daemon-reload

需要注意的是启动的时候需要注意：systemctl start redis@7003

  1. # systemctl status redis@7003

  2. ● redis@7003.service - Redis-7003

  3.   Loaded: loaded (/etc/systemd/system/redis@.service; disabled; vendor preset: disabled)

  4.   Active: active (running) since Fri 2019-04-05 10:45:27 CST; 7s ago

  5.  Process: 20434 ExecStart=/usr/local/redis/bin/redis-server /usr/local/redis/config/%i/redis.conf (code=exited, status=0/SUCCESS)

  6. Main PID: 20435 (redis-server)

  7.   CGroup: /system.slice/system-redis.slice/redis@7003.service

  8.           └─20435 /usr/local/redis/bin/redis-server 192.168.152.131:7003 [cluster]

  10. Apr 05 10:45:27 master systemd[1]: Starting Redis-7003...

  11. Apr 05 10:45:27 master systemd[1]: PID 20048 read from file /var/run/redis_7003.pid does not exist or is a zombie.

  12. Apr 05 10:45:27 master systemd[1]: Started Redis-7003.

 

创建集群

随意找一台机器执行此命令

  1. /usr/local/redis/bin/redis-cli --cluster create 192.168.152.129:7000 192.168.152.129:7001 192.168.152.131:7002 192.168.152.131:7003 192.168.152.130:7004 192.168.152.130:7005 --cluster-replicas 1

执行后的输出如下：（输入yes）

最后输出：[OK] All 16384 slots covered

验证

  1. 随便连接一个节点

  2. redis-cli -h 192.168.152.129 -p 7000

  3. > set name afei

  5. 切到其他节点查看

  6. redis-cli -h 192.168.152.129 -p 7001

  7. > get name

 

查看主节点

  1. # ./redis-cli -h 192.168.152.131 -p 7002 cluster nodes | grep master

  2. 31e0e0d8420fd2c47f23c584fcfcc7666942b41a 192.168.152.129:7000@17000 master - 0 1554428553356 1 connected 0-5460

  3. d4e934df845735259d5d47204452df3d9089ee0c 192.168.152.130:7004@17004 master - 0 1554428552549 5 connected 10923-16383

  4. 35e3bc663c1d393c13ace6cb9d9b3d54ff18961d 192.168.152.131:7002@17002 myself,master - 0 1554428551000 3 connected 5461-10922

增加与删除节点

  1. 增加节点

  2. # /usr/local/redis/bin/redis-cli --cluster add-node 192.168.152.131:7003 192.168.152.131:7002

  3. >>> Adding node 192.168.152.131:7003 to cluster 192.168.152.131:7002

  4. >>> Performing Cluster Check (using node 192.168.152.131:7002)

  5. M: 35e3bc663c1d393c13ace6cb9d9b3d54ff18961d 192.168.152.131:7002

  6.   slots:[5461-10922] (5462 slots) master

  7.   1 additional replica(s)

  8. M: 31e0e0d8420fd2c47f23c584fcfcc7666942b41a 192.168.152.129:7000

  9.   slots:[0-5460] (5461 slots) master

  10. M: d4e934df845735259d5d47204452df3d9089ee0c 192.168.152.130:7004

  11.   slots:[10923-16383] (5461 slots) master

  12.   1 additional replica(s)

  13. S: e1686d5ce576e9446211fb9ae82999f4e98a8ad5 192.168.152.129:7001

  14.   slots: (0 slots) slave

  15.   replicates d4e934df845735259d5d47204452df3d9089ee0c

  16. S: 4b78d60924bfa51d975d7129fe71a15c1690e8a4 192.168.152.130:7005

  17.   slots: (0 slots) slave

  18.   replicates 35e3bc663c1d393c13ace6cb9d9b3d54ff18961d

  19. [OK] All nodes agree about slots configuration.

  20. >>> Check for open slots...

  21. >>> Check slots coverage...

  22. [OK] All 16384 slots covered.

  23. >>> Send CLUSTER MEET to node 192.168.152.131:7003 to make it join the cluster.

  24. [OK] New node added correctly.

  27. 删除节点

  28. # /usr/local/redis/bin/redis-cli --cluster del-node 192.168.152.131:7003 6185115d72a1f978f78334077aecb33ec52ef313

  29. >>> Removing node 6185115d72a1f978f78334077aecb33ec52ef313 from cluster 192.168.152.131:7003

  30. >>> Sending CLUSTER FORGET messages to the cluster...

  31. >>> SHUTDOWN the node.

如果增加指定的节点为slave的话在后面加–cluster-slave

问题记录

1、错误信息：slave,fail,noaddr disconnected

  1. # /usr/local/redis/bin/redis-cli -h 192.168.152.131 -p 7002 cluster nodes

  2. f37660309eedf303be2e1764f8721425ad2344fa :0@0 slave,fail,noaddr 31e0e0d8420fd2c47f23c584fcfcc7666942b41a 1554431564423 1554431562000 4 disconnected

  3. 31e0e0d8420fd2c47f23c584fcfcc7666942b41a 192.168.152.129:7000@17000 master - 0 1554433711000 1 connected 0-5460

  4. d4e934df845735259d5d47204452df3d9089ee0c 192.168.152.130:7004@17004 master - 0 1554433711000 5 connected 10923-16383

  5. e1686d5ce576e9446211fb9ae82999f4e98a8ad5 192.168.152.129:7001@17001 slave d4e934df845735259d5d47204452df3d9089ee0c 0 1554433711531 5 connected

  6. 4b78d60924bfa51d975d7129fe71a15c1690e8a4 192.168.152.130:7005@17005 slave 35e3bc663c1d393c13ace6cb9d9b3d54ff18961d 0 1554433711733 6 connected

  7. 35e3bc663c1d393c13ace6cb9d9b3d54ff18961d 192.168.152.131:7002@17002 myself,master - 0 1554433709000 3 connected 5461-10922

解决办法：

  1. /usr/local/redis/bin/redis-cli -h 192.168.152.131 -p 7002 cluster forget f37660309eedf303be2e1764f8721425ad2344fa

重新增加即可。

2、错误信息：Wrong number of arguments for specified –cluster sub command

  1. # /usr/local/redis/bin/redis-cli --cluster add-node 192.168.152.131:7003 --cluster-slave

  2. [ERR] Wrong number of arguments for specified --cluster sub command

这个错是由于命令输错了，看下帮助信息才知道语法写错了，帮助信息如下：

  1. # ./redis-cli --cluster help

  2. Cluster Manager Commands:

  3.  create         host1:port1 ... hostN:portN

  4.                 --cluster-replicas <arg>

  5.  check          host:port

  6.                 --cluster-search-multiple-owners

  7.  info           host:port

  8.  fix            host:port

  9.                 --cluster-search-multiple-owners

  10.  reshard        host:port

  11.                 --cluster-from <arg>

  12.                 --cluster-to <arg>

  13.                 --cluster-slots <arg>

  14.                 --cluster-yes

  15.                 --cluster-timeout <arg>

  16.                 --cluster-pipeline <arg>

  17.                 --cluster-replace

  18.  rebalance      host:port

  19.                 --cluster-weight <node1=w1...nodeN=wN>

  20.                 --cluster-use-empty-masters

  21.                 --cluster-timeout <arg>

  22.                 --cluster-simulate

  23.                 --cluster-pipeline <arg>

  24.                 --cluster-threshold <arg>

  25.                 --cluster-replace

  26.  add-node       new_host:new_port existing_host:existing_port

  27.                 --cluster-slave

  28.                 --cluster-master-id <arg>

  29.  del-node       host:port node_id

  30.  call           host:port command arg arg .. arg

  31.  set-timeout    host:port milliseconds

  32.  import         host:port

  33.                 --cluster-from <arg>

  34.                 --cluster-copy

  35.                 --cluster-replace

  36.  help

add-node后要跟两个地址，一个新的，一个已经存在的。修改后重新执行可以

  1. # /usr/local/redis/bin/redis-cli --cluster add-node 192.168.152.131:7003 192.168.152.131:7002

  2. >>> Adding node 192.168.152.131:7003 to cluster 192.168.152.131:7002

  3. >>> Performing Cluster Check (using node 192.168.152.131:7002)

  4. M: 35e3bc663c1d393c13ace6cb9d9b3d54ff18961d 192.168.152.131:7002

  5.   slots:[5461-10922] (5462 slots) master

  6.   1 additional replica(s)

  7. M: 31e0e0d8420fd2c47f23c584fcfcc7666942b41a 192.168.152.129:7000

  8.   slots:[0-5460] (5461 slots) master

  9. M: d4e934df845735259d5d47204452df3d9089ee0c 192.168.152.130:7004

  10.   slots:[10923-16383] (5461 slots) master

  11.   1 additional replica(s)

  12. S: e1686d5ce576e9446211fb9ae82999f4e98a8ad5 192.168.152.129:7001

  13.   slots: (0 slots) slave

  14.   replicates d4e934df845735259d5d47204452df3d9089ee0c

  15. S: 4b78d60924bfa51d975d7129fe71a15c1690e8a4 192.168.152.130:7005

  16.   slots: (0 slots) slave

  17.   replicates 35e3bc663c1d393c13ace6cb9d9b3d54ff18961d

  18. [OK] All nodes agree about slots configuration.

  19. >>> Check for open slots...

  20. >>> Check slots coverage...

  21. [OK] All 16384 slots covered.

  22. >>> Send CLUSTER MEET to node 192.168.152.131:7003 to make it join the cluster.

  23. [OK] New node added correctly.

3、问题信息：Node 192.168.152.131:7003 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

  1. # /usr/local/redis/bin/redis-cli --cluster add-node 192.168.152.131:7003 192.168.152.131:7002 --cluster-slave

  2. >>> Adding node 192.168.152.131:7003 to cluster 192.168.152.131:7002

  3. >>> Performing Cluster Check (using node 192.168.152.131:7002)

  4. M: 35e3bc663c1d393c13ace6cb9d9b3d54ff18961d 192.168.152.131:7002

  5.   slots:[5461-10922] (5462 slots) master

  6.   1 additional replica(s)

  7. M: 31e0e0d8420fd2c47f23c584fcfcc7666942b41a 192.168.152.129:7000

  8.   slots:[0-5460] (5461 slots) master

  9. M: d4e934df845735259d5d47204452df3d9089ee0c 192.168.152.130:7004

  10.   slots:[10923-16383] (5461 slots) master

  11.   1 additional replica(s)

  12. S: e1686d5ce576e9446211fb9ae82999f4e98a8ad5 192.168.152.129:7001

  13.   slots: (0 slots) slave

  14.   replicates d4e934df845735259d5d47204452df3d9089ee0c

  15. S: 4b78d60924bfa51d975d7129fe71a15c1690e8a4 192.168.152.130:7005

  16.   slots: (0 slots) slave

  17.   replicates 35e3bc663c1d393c13ace6cb9d9b3d54ff18961d

  18. [OK] All nodes agree about slots configuration.

  19. >>> Check for open slots...

  20. >>> Check slots coverage...

  21. [OK] All 16384 slots covered.

  22. Automatically selected master 192.168.152.129:7000

  23. [ERR] Node 192.168.152.131:7003 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

网上参考的解决办法：

  1. remove the aof/rdb backup file

  2. remove the node_conf file

  3. flushdb if necessary


redis
文章转载自feelwow，如果涉嫌侵权，请发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。

评论