
本文首发于Ressmix个人站点:https://www.tpvlog.com
对于一个能够支撑超高并发的大型分布式系统来说,像Redis这类分布式缓存是必不可少。Redis在单机部署的模式下,QPS几乎不可能超过10万+,除非机器配置特别好且Redis操作不太复杂。
我们知道,对于数据库来说,如果想要提升读写性能,最简单的方式就是做一主多从+读写分离。对于分布式缓存也是一样的道理,因为缓存一般都是用来支撑读请求的高并发,写请求相对较少(一般也就每秒一两千写请求),所以非常适合读写分离的架构。
关于Redis的复制原理,我在进阶篇的《分布式框架之高性能:Redis主从同步》已经详细讲解过了,不熟悉的读者可以先去了解下。
一、主从架构搭建
在生产环境下,我们必须要将Master节点的持久化功能打开,否则万一Master宕机后重启,此时Slave连上Master后,会触发一次全量复制,master就会将空的数据集同步到slave上去,导致Slave中的数据也被清空。
我们先来搭建一个一主二从的Redis构架,我首先在ressmix-dsf02这个节点上安装单机版本的Redis。具体的安装步骤不再赘述,读者可以参考我在《Redis持久化实战》中讲解的搭建步骤。
1.1 配置步骤
搭建完Redis节点后,我们按照以下步骤进行读写分离的配置,ressmix-dsf01作为Master,ressmix-dsf02和ressmix-dsf03作为Slave。(我这里只操作ressmix-dsf02,ressmix-dsf03读者可以自行操作)
修改Slave节点的配置文件,配置
replicaof ressmix-dsf01 6379
,这样ressmix-dsf01节点就作为了ressmix-dsf02的Master节点;强制读写分离:修改Slave节点的配置文件,配置
replica-read-only yes
,这样Slave节点会拒绝所有的写操作(Redis 2.6以后Slave节点默认就是只读的,所以这个版本以后的Redis默认可以不设置);集群安全认证:修改Slave节点的配置文件,配置
masterauth ressmix
,其中ressmix是我设置的认证密码;停止Master节点,然后修改Master节点的配置文件,配置
requirepass ressmix
;主从节点均配置
appendonly yes
,开启AOF持久化;绑定节点IP:修改Slave节点的配置文件,配置
bind 192.168.0.109
,其中192.168.0.109为ressmix-dsf02的IP,同理也把Master节点的这个配置修改下。另外,为了以防万一,每个节点都执行下iptables -A INPUT -ptcp --dport 6379 -j ACCEPT
,用于放开6379端口,然后清理下防火墙:sudo iptables -F
。
上述操作全部配置完成后,我们通过以下命令启动主节点,然后以相同方式启动从节点ressmix-dsf02。
1cd /etc/init.d
2./redis_6379 start
节点启动后,我们可以先在Master节点中写入一条记录:
1redis-cli -h 192.168.0.107 -a ressmix
2set k1 v1

然后在ressmix-dsf02节点可以查看到同步过来的数据:

我们可以通过执行info replication
查看主从复制的状态:

二、性能压测
搭建完Redis的主从架构后,可以对其做一个基准压测,测一下Redis的性能和QPS。Redis自身提供了redis-benchmark压测工具,可以用于一些简单场景下性能测试。
压测工具位于redis安装包的src目录下:
1./redis-benchmark -h 192.168.0.107
常用参数如下:
1-c <clients> Number of parallel connections (default 50)
2-n <requests> Total number of requests (default 100000)
3-d <size> Data size of SET/GET value in bytes (default 2)
我们可以根据自己系统高峰期的业务量来设置参数,比如在高峰期,瞬时最大用户量会达到10万,总请求数为1000万,每条数据的大小为50字节,则可以像下面这样模拟请求:
1./redis-benchmark -h 192.168.0.107 -c 100000 -n 10000000 -d 50
压测的结果可能像下面这样,显示了不同操作的每秒请求数:
1====== PING_INLINE ======
2 100000 requests completed in 1.28 seconds
3 50 parallel clients
4 3 bytes payload
5 keep alive: 1
6
799.78% <= 1 milliseconds
899.93% <= 2 milliseconds
999.97% <= 3 milliseconds
10100.00% <= 3 milliseconds
1178308.54 requests per second
12
13====== PING_BULK ======
14 100000 requests completed in 1.30 seconds
15 50 parallel clients
16 3 bytes payload
17 keep alive: 1
18
1999.87% <= 1 milliseconds
20100.00% <= 1 milliseconds
2176804.91 requests per second
22
23====== SET ======
24 100000 requests completed in 2.50 seconds
25 50 parallel clients
26 3 bytes payload
27 keep alive: 1
28
295.95% <= 1 milliseconds
3099.63% <= 2 milliseconds
3199.93% <= 3 milliseconds
3299.99% <= 4 milliseconds
33100.00% <= 4 milliseconds
3440032.03 requests per second
35
36====== GET ======
37 100000 requests completed in 1.30 seconds
38 50 parallel clients
39 3 bytes payload
40 keep alive: 1
41
4299.73% <= 1 milliseconds
43100.00% <= 2 milliseconds
44100.00% <= 2 milliseconds
4576628.35 requests per second
46
47====== INCR ======
48 100000 requests completed in 1.90 seconds
49 50 parallel clients
50 3 bytes payload
51 keep alive: 1
52
5380.92% <= 1 milliseconds
5499.81% <= 2 milliseconds
5599.95% <= 3 milliseconds
5699.96% <= 4 milliseconds
5799.97% <= 5 milliseconds
58100.00% <= 6 milliseconds
5952548.61 requests per second
60
61====== LPUSH ======
62 100000 requests completed in 2.58 seconds
63 50 parallel clients
64 3 bytes payload
65 keep alive: 1
66
673.76% <= 1 milliseconds
6899.61% <= 2 milliseconds
6999.93% <= 3 milliseconds
70100.00% <= 3 milliseconds
7138684.72 requests per second
72
73====== RPUSH ======
74 100000 requests completed in 2.47 seconds
75 50 parallel clients
76 3 bytes payload
77 keep alive: 1
78
796.87% <= 1 milliseconds
8099.69% <= 2 milliseconds
8199.87% <= 3 milliseconds
8299.99% <= 4 milliseconds
83100.00% <= 4 milliseconds
8440469.45 requests per second
85
86====== LPOP ======
87 100000 requests completed in 2.26 seconds
88 50 parallel clients
89 3 bytes payload
90 keep alive: 1
91
9228.39% <= 1 milliseconds
9399.83% <= 2 milliseconds
94100.00% <= 2 milliseconds
9544306.60 requests per second
96
97====== RPOP ======
98 100000 requests completed in 2.18 seconds
99 50 parallel clients
100 3 bytes payload
101 keep alive: 1
102
10336.08% <= 1 milliseconds
10499.75% <= 2 milliseconds
105100.00% <= 2 milliseconds
10645871.56 requests per second
107
108====== SADD ======
109 100000 requests completed in 1.23 seconds
110 50 parallel clients
111 3 bytes payload
112 keep alive: 1
113
11499.94% <= 1 milliseconds
115100.00% <= 2 milliseconds
116100.00% <= 2 milliseconds
11781168.83 requests per second
118
119====== SPOP ======
120 100000 requests completed in 1.28 seconds
121 50 parallel clients
122 3 bytes payload
123 keep alive: 1
124
12599.80% <= 1 milliseconds
12699.96% <= 2 milliseconds
12799.96% <= 3 milliseconds
12899.97% <= 5 milliseconds
129100.00% <= 5 milliseconds
13078369.91 requests per second
131
132====== LPUSH (needed to benchmark LRANGE) ======
133 100000 requests completed in 2.47 seconds
134 50 parallel clients
135 3 bytes payload
136 keep alive: 1
137
13815.29% <= 1 milliseconds
13999.64% <= 2 milliseconds
14099.94% <= 3 milliseconds
141100.00% <= 3 milliseconds
14240420.37 requests per second
143
144====== LRANGE_100 (first 100 elements) ======
145 100000 requests completed in 3.69 seconds
146 50 parallel clients
147 3 bytes payload
148 keep alive: 1
149
15030.86% <= 1 milliseconds
15196.99% <= 2 milliseconds
15299.94% <= 3 milliseconds
15399.99% <= 4 milliseconds
154100.00% <= 4 milliseconds
15527085.59 requests per second
156
157====== LRANGE_300 (first 300 elements) ======
158 100000 requests completed in 10.22 seconds
159 50 parallel clients
160 3 bytes payload
161 keep alive: 1
162
1630.03% <= 1 milliseconds
1645.90% <= 2 milliseconds
16590.68% <= 3 milliseconds
16695.46% <= 4 milliseconds
16797.67% <= 5 milliseconds
16899.12% <= 6 milliseconds
16999.98% <= 7 milliseconds
170100.00% <= 7 milliseconds
1719784.74 requests per second
172
173====== LRANGE_500 (first 450 elements) ======
174 100000 requests completed in 14.71 seconds
175 50 parallel clients
176 3 bytes payload
177 keep alive: 1
178
1790.00% <= 1 milliseconds
1800.07% <= 2 milliseconds
1811.59% <= 3 milliseconds
18289.26% <= 4 milliseconds
18397.90% <= 5 milliseconds
18499.24% <= 6 milliseconds
18599.73% <= 7 milliseconds
18699.89% <= 8 milliseconds
18799.96% <= 9 milliseconds
18899.99% <= 10 milliseconds
189100.00% <= 10 milliseconds
1906799.48 requests per second
191
192====== LRANGE_600 (first 600 elements) ======
193 100000 requests completed in 18.56 seconds
194 50 parallel clients
195 3 bytes payload
196 keep alive: 1
197
1980.00% <= 2 milliseconds
1990.23% <= 3 milliseconds
2001.75% <= 4 milliseconds
20191.17% <= 5 milliseconds
20298.16% <= 6 milliseconds
20399.04% <= 7 milliseconds
20499.83% <= 8 milliseconds
20599.95% <= 9 milliseconds
20699.98% <= 10 milliseconds
207100.00% <= 10 milliseconds
2085387.35 requests per second
209
210====== MSET (10 keys) ======
211 100000 requests completed in 4.02 seconds
212 50 parallel clients
213 3 bytes payload
214 keep alive: 1
215
2160.01% <= 1 milliseconds
21753.22% <= 2 milliseconds
21899.12% <= 3 milliseconds
21999.55% <= 4 milliseconds
22099.70% <= 5 milliseconds
22199.90% <= 6 milliseconds
22299.95% <= 7 milliseconds
223100.00% <= 8 milliseconds
22424869.44 requests per second
三、总结
本章,我重点讲解了如何进行生产环境的Redis读写分离部署,读者可以自己尝试在虚拟机中动手进行节点部署,以加深印象。




