04.CBase-EC Achieving Optimal Throughput-Storage Efficiency Trade-Off Using Erasure Codes.pdf
免费下载
electronics
Article
CBase-EC: Achieving Optimal Throughput-Storage Efficiency
Trade-Off Using Erasure Codes
Chuqiao Xiao * , Yefeng Xia , Qian Zhang , Xueqing Gong * and Liyan Zhu
Citation: Xiao, C.; Xia, Y.; Zhang, Q.;
Gong, X.; Zhu, L. CBase-EC:
Achieving Optimal Throughput-
Storage Efficiency Trade-Off Using
Erasure Codes. Electronics 2021, 10,
126. https://doi.org/10.3390/
electronics10020126
Received: 12 November 2020
Accepted: 5 January 2021
Published: 8 January 2021
Publisher’s Note: MDPI stays neu-
tral with regard to jurisdictional clai-
ms in published maps and institutio-
nal affiliations.
Copyright: © 2021 by the authors. Li-
censee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and con-
ditions of the Creative Commons At-
tribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Software Engineering Institute, East China Normal University, Shanghai 200062, China;
51184501163@stu.ecnu.edu.cn (Y.X.); 52184501012@stu.ecnu.edu.cn (Q.Z.); 51184501093@stu.ecnu.edu.cn (L.Z.)
* Correspondence: 52184501011@stu.ecnu.edu.cn (C.X.); xqgong@sei.ecnu.edu.cn (X.G.)
Abstract:
Many distributed database systems that guarantee high concurrency and scalability adopt
read-write separation architecture. Simultaneously, these systems need to store massive amounts
of data daily, requiring different mechanisms for storing and accessing data, such as hot and cold
data access strategies. Unlike distributed storage systems, the distributed database splits a table into
sub-tables or shards, and the request frequency of each sub-table is not the same within a specific
time. Therefore, it is not only necessary to design hot-to-cold approaches to reduce storage overhead,
but also cold-to-hot methods to ensure high concurrency of those systems. We present a new
redundant strategy named CBase-EC, using erasure codes to trade the performances of transaction
processing and storage efficiency for CBase database systems developed for financial scenarios of
the Bank. Two algorithms are proposed: the hot-cold tablets (shards) recognition algorithm and the
hot-cold dynamic conversion algorithm. Then we adopt two optimization approaches to improve
CBase-EC performance. In the experiment, we compare CBase-EC with three-replicas in CBase. The
experimental results show that although the transaction processing performance declined by no more
than 6%, the storage efficiency increased by 18.4%.
Keywords: erasure codes; distributed database system; hot and cold separation; storage efficiency
1. Introduction
With the increasing complexity of the Internet business model, various Distributed
Database Management System (DDBMS) architectures are emerging and developing. Re-
lational DDBMSs have always been adopted by the master/slaver read-write separation
architecture adaptive for large-scale and highly concurrent business scenarios [
1
]. More-
over, DDBMSs store massive and various data on many commodity servers daily, and the
data access mechanisms and redundancy strategies of different data need dividing [2]. In
any distributed system, whether it is a DDBMS or a distributed storage system, system
reliability needs to be guaranteed. The reliability of data storage is ensured in part by
adopting redundancy in some form, such as simple replication or a more sophisticated
erasure code (EC) strategy. Some DDBMSs always use the complete data backup strategy,
also known as multi-replicas strategy, to guarantee high system reliability, but they have
low storage utilization [
3
]. Nevertheless, due to the requirements of business and user data
integrity, a large amount of data must be stored and cannot be deleted [
4
]. As time goes on,
a significant fraction of data stored in DDBMSs is rarely accessed. These data are named
cold data [
5
]. Cold data have been identified as the fastest-growing storage segment, with
a 60% cumulative annual growth rate [6].
At present, to deal with the unavailability, or loss, of data caused by error failure,
existing disk array storage systems and distributed storage systems often use EC strategies,
which can tolerate broader classes of failure scenarios with less extra storage overhead [
7
].
Disk array storage, such as the redundant array of independent disks (RAID) organizes
multiple independent storage devices (HDD, SSD) into a logically continuous storage space
Electronics 2021, 10, 126. https://doi.org/10.3390/electronics10020126 https://www.mdpi.com/journal/electronics
Electronics 2021, 10, 126 2 of 16
to provide the system with larger storage space. In distributed storage systems, the design
of erasure coding technology has great practical significance. These studies included the
following aspects: trading off storage efficiency and repair bandwidth overhead, improving
recovery rates, selecting the optimal data block storage location, and optimizing utilization
of CPU resources.
However, data read and write performances of ECs are not as good as the multi-
replicas strategies on DDBMSs. The ECs are not suitable for DDBMSs for two reasons
from the perspective of query and update. First, transaction processing requires frequent
data access in DDBMSs. In the read-write separation architecture, the multi-replicas
mechanism can improve system throughput. Second, after completing the data update on
a master node, only the log needs to be transmitted to other slaver backup nodes to the
data asynchronous update. Although ECs are not as efficient as multi-replicas strategies in
terms of data access and data updates, it can significantly reduce storage overhead and
improve storage utilization [8].
The CBase (CBase Homepage: https://github.com/BankOfCommunications/CBASE.)
is a high availability distributed relational database developed for financial scenarios. It real-
izes cross-record and cross-table transactions on hundreds of terabytes (TB) of data and hun-
dreds of billions of records. The CBase adopts a distributed architecture with read-write sep-
aration based on OceanBase 0.4.2 (OceanBase Homepage: https://oceanbase.alipay.com/),
and its redundancy strategy is the three-replicas. It divides the data into baseline data
and incremental data and merges these data at a specific period. This system architecture
is representative of the research prototype, so relevant technical work on CBase can be
migrated and extended to other database systems.
Consequently, we trade off the transactions processing performance and storage
efficiency, and propose a new redundancy strategy CBase-EC, which can recognize and
dynamically convert the hot and cold tablets of CBase.
Our contributions:
1.
We propose the redundancy strategy CBase-EC, which includes the hot and cold
tablets recognition algorithm, the hot and cold tablets dynamic conversion algorithm
and the load balance scheme.
2.
The encoding and updating performances of CBase-EC based on locally repairable
codes (LRC) have been reduced, and we have designed the LRC increment the
update algorithm to optimize the updating process and have presented a heuristic
algorithm to find local optimal Bitmatrix of LRC to improve encoding and decoding
performances.
3.
We use sysBench, a benchmark tool in the distributed database field, to compare
the storage efficiency and transaction processing performance of CBase-EC with the
original three-replicas strategy on the CBase database system. The experimental
results show that the strategy presented in this paper has no significant decrease
in transaction throughput, at most about 6%, but the storage efficiency improves
by 18.4%.
The remainder of this paper is organized as follows. In Section 2, we introduce some
basic concepts including the comparison between EC and the replication strategy and
basic concepts about the CBase database system. In Sections 3 and 4, we propose the new
redundancy strategy CBase-EC and introduce two optimization methods of encoding and
updating. In Section 5, we report on our comparative experiments, mainly comparing the
storage efficiency and transaction processing performances for new CBase with the original
CBase. The conclusion and future works are in Section 6.
2. Background
In this section, we introduce the technical terms of EC and compare the EC with the
replication strategy. Then we briefly describe the query and update process of the CBase
database system.
of 17
免费下载
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文档的来源（墨天轮），文档链接，文档作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。
下载排行榜
评论