09.Fast Log Replication in Highly Available Data Store.pdf
免费下载
Fast Log Replication in Highly Available
Data Store
Donghui Wang
1
, Peng Cai
1(
B
)
, Weining Qian
1,2
, Aoying Zhou
1
, Tianze Pang
2
,
and Jing Jiang
2
1
Institute for Data Science and Engineering, East China Normal University,
Shanghai 200062, People’s Republic of China
donghuiwang@stu.ecnu.edu.cn, {pcai,wnqian,ayzhou}@sei.ecnu.edu.cn
2
Software Development Center, Bank of Communications, Shanghai 201201,
People’s Republic of China
{pangtz,jiangj5}@bankcomm.com
Abstract. Modern large-scale data stores widely adopt consensus pro-
tocols to achieve high availability and throughput. The recently proposed
Raft algorithm has better understandability and widely implemented
in large amount of open source projects. In these consensus algorithms
including Raft, log replication is a common and frequently used operation
which has significant impact on the system performance. Especially, since
the commit latency is capped by the slowest follower out of the majority
followers responded to the leader, it’s important to design a fast scheme
to process the replicated logs by follower nodes. Based on the analysis on
how the follower node handles the received log entries in Raft algorithm,
we figure out the main factors influencing the duration time from when
the follower receives the log and to when it acknowledges the leader this
log was received. In terms of these factors we propose an effective log
replication scheme to optimize the process of flushing logs to disk and
replaying them, referred to as Raft with Fast Followers (FRaft). Finally,
we compare the performance of Raft and FRaft using YCSB benchmark
and Sysbench test tools, and experimental results demonstrate FRaft has
lower latency and higher throughput than the Raft only using straight-
forward pipeline and batch optimization for log replication.
Keywords: Log replication
· High availability · Consensus system ·
Raft
1 Introduction
Today’s modern applications often require the back-end data store not only to
provide the acceptable system performance but also to meet the high avail-
ability requirements. State machine replication is regarded as the most general
approach to implementing a highly available data store where the data is repli-
cated across a set of servers and consensus protocols are used to guarantee the
consistency among different copies. Consensus protocols, including Paxos or its
c
Springer International Publishing AG 2017
L. Chen et al. (Eds.): APWeb-WAIM 2017, Part II, LNCS 10367, pp. 245–259, 2017.
DOI: 10.1007/978-3-319-63564-4
20
246 D. Wang et al.
variants [11,17,18], Viewstamped Replication [19] and Zab [15], reach an agree-
ment on each operation and ensure all replicas execute the operation in the same
order. Consensus is the fundamental problem in distributed systems and these
protocols have became the key component of large-scale and fault-tolerant data
store [6,10,21].
In contrast to the famous Paxos protocol, the recently proposed Raft algo-
rithm has better understandability and widely implemented in large amount of
open source projects [2,20]. During the execution of these consensus protocols
including the recently proposed Raft, log replication is a common and frequently
used operation which has significant impact on the system performance. In Raft,
a transaction can be committed if its log has been replicated on the majority of
followers. However, log replication algorithm also comes with inevitable perfor-
mance problems because of the latency caused by network and processing time
in followers (mainly from disk latency).
Raft achieves consensus among a group of members via an elected leader.
Only leader can accept new request from clients, and then replicates log entries
to followers. When the leader accept the acknowledgment from the majority of
followers, it commits the transaction and both leader and followers can safely
apply log entries to their replicas. In the naive implementation of Raft, the leader
propagates one request at a time. In general, this is highly ineffective because
multiple network transmissions increase the delay of each request. There are two
optimizations widely used in the implementation of consensus protocols [5,7,
12,13]: batching and pipeline. Batching is to pack several requests into a single
AppendEntries RPC, which spread the overhead on a set of requests. Pipeline
allows the leader to propagate a new AppendEntries RPC to followers before
the previous ones are acknowledged [16]. Pipeline can effectively improve the
throughput especially in the WAN network with high latency.
Although batching and pipeline can improve the performance of Raft con-
sensus protocol, the follower still needs to wait for flushing a batch to disk before
processing the next batch in the task queue which holds the many batches sent
by the leader. On the other hand, the strategy of replaying logs after they are
committed incurs a large amount of expensive memory copy operation (see the
details in the problem analysis section). To address these challenges, we redesign
the log replication scheme for Raft protocol. The basic idea is to separate flushing
a batch log from the log processing flow. Instead of directly writing the received
batch logs to disk by followers, the batch is immediately moved from the task
queue to a batch buffer. By this way, the next batch can be handled without
any blocking. A single thread is used to monitor the batch buffer, and asyn-
chronously flush a group of batch to disk in order to reduce disk IO overhead.
Furthermore, in order to decrease the operations of memory copy, the received
logs are also replayed immediately but the applied results are invisible until the
corresponding transaction are committed.
The time consuming on processing the logs by follower has significant impact
on the throughput and the end-to-end transaction response time as perceived by
the user. In this paper, we optimize the log flushing and replay in the follower,
of 15
免费下载
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文档的来源（墨天轮），文档链接，文档作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。
下载排行榜
评论