暂无图片
暂无图片
暂无图片
暂无图片
暂无图片

3.可扩展性和弹性:集群,节点和分片

产品与码农 2020-04-02
259

Scalability and resilience: clusters, nodes, and shards

  • Elasticsearch是为满足您的需要而构建的。这是通过天生具有分布式来实现的。您可以向集群中添加服务器(节点)来增加容量,Elasticsearch会自动将您的数据和查询负载分布到所有可用节点上。无需彻底修改您的应用程序,Elasticsearch知道如何平衡多节点集群以提供伸缩性和高可用性。节点越多越好。Elasticsearch is built to be always available and to scale with your needs. It does this by being distributed by nature. You can add servers (nodes) to a cluster to increase capacity and Elasticsearch automatically distributes your data and query load across all of the available nodes. No need to overhaul your application, Elasticsearch knows how to balance multi-node clusters to provide scale and high availability. The more nodes, the merrier.

  • 这是如何运作的?在幕后,Elasticsearch索引实际上只是一个或多个物理碎片的逻辑分组,其中每个碎片实际上是一个独立的索引。通过将文档分布在多个分片中的索引中,然后将这些分片分布在多个节点中,Elasticsearch可以确保冗余,这既可以防止硬件故障,又可以在将节点添加到集群中时提高查询能力。随着集群的增长(或收缩),Elasticsearch会自动迁移碎片以重新平衡集群。How does this work? Under the covers, an Elasticsearch index is really just a logical grouping of one or more physical shards, where each shard is actually a self-contained index. By distributing the documents in an index across multiple shards, and distributing those shards across multiple nodes, Elasticsearch can ensure redundancy, which both protects against hardware failures and increases query capacity as nodes are added to a cluster. As the cluster grows (or shrinks), Elasticsearch automatically migrates shards to rebalance the cluster.

  • 分片有两种类型:主数据库和副本数据库。索引中的每个文档都属于一个主分片。副本分片是主分片的副本。副本可提供数据的冗余副本,以防止硬件故障并提高处理读取请求(例如搜索或检索文档)的能力。There are two types of shards: primaries and replicas. Each document in an index belongs to one primary shard. A replica shard is a copy of a primary shard. Replicas provide redundant copies of your data to protect against hardware failure and increase capacity to serve read requests like searching or retrieving a document.

  • 创建索引时,索引中主碎片的数量是固定的,但是副本碎片的数量可以随时更改,而不会中断索引或查询操作。The number of primary shards in an index is fixed at the time that an index is created, but the number of replica shards can be changed at any time, without interrupting indexing or query operations.

这取决于... ...It depends…
  • 在分片大小和为索引配置的主分片数量方面,存在许多性能方面的考虑和权衡取舍。碎片越多,维护这些索引的开销就越大。分片大小越大,当Elasticsearch需要重新平衡集群时,移动分片所需的时间就越长。There are a number of performance considerations and trade offs with respect to shard size and the number of primary shards configured for an index. The more shards, the more overhead there is simply in maintaining those indices. The larger the shard size, the longer it takes to move shards around when Elasticsearch needs to rebalance a cluster.

  • 查询很多小的分片会使每个分片的处理速度更快,但是更多的查询意味着更多的开销,因此查询较小数量的大分片可能会更快。简而言之...要视情况而定。Querying lots of small shards makes the processing per shard faster, but more queries means more overhead, so querying a smaller number of larger shards might be faster. In short…it depends.

  • 作为起点:As a starting point:

    • 旨在将平均分片大小保持在几GB到几十GB之间。对于具有基于时间的数据的用例,通常会看到20GB到40GB范围内的碎片。Aim to keep the average shard size between a few GB and a few tens of GB. For use cases with time-based data, it is common to see shards in the 20GB to 40GB range.

    • 避免庞大的碎片问题。节点可以容纳的分片数量与可用堆空间成比例。通常,每GB堆空间中的分片数量应少于20。Avoid the gazillion shards problem. The number of shards a node can hold is proportional to the available heap space. As a general rule, the number of shards per GB of heap space should be less than 20.

  • 确定用例最佳配置的最佳方法是通过 使用自己的数据和查询进行测试。The best way to determine the optimal configuration for your use case is through testing with your own data and queries.

在灾难的情况下In case of disaster
  • 出于性能原因,群集内的节点必须位于同一网络上。跨不同数据中心中的节点的群集中的碎片平衡太简单了。但是高可用性架构要求您避免将所有鸡蛋都放在一个篮子里。如果一个位置发生重大故障,则另一个位置的服务器需要能够无缝接管。这就需要:跨集群复制(CCR)。For performance reasons, the nodes within a cluster need to be on the same network. Balancing shards in a cluster across nodes in different data centers simply takes too long. But high-availability architectures demand that you avoid putting all of your eggs in one basket. In the event of a major outage in one location, servers in another location need to be able to take over. Seamlessly. The answer? Cross-cluster replication (CCR).

  • CCR提供了一种方法,可以自动将索引从主群集同步到可以用作热备份的辅助远程群集。如果主群集出现故障,则辅助群集可以接管。您还可以使用CCR创建辅助群集,以接近地理位置的方式向您的用户提供读取请求。CCR provides a way to automatically synchronize indices from your primary cluster to a secondary remote cluster that can serve as a hot backup. If the primary cluster fails, the secondary cluster can take over. You can also use CCR to create secondary clusters to serve read requests in geo-proximity to your users.

  • 跨集群复制是主动-被动的。主群集上的索引是活动的领导者索引,并处理所有写请求。复制到辅助群集的索引是只读关注者。Cross-cluster replication is active-passive. The index on the primary cluster is the active leader index and handles all write requests. Indices replicated to secondary clusters are read-only followers.

Care and feeding
  • 与任何企业系统一样,您需要工具来保护,管理和监视Elasticsearch集群。集成到Elasticsearch中的安全性,监视和管理功能使您可以将Kibana 用作控制中心来管理集群。类似的特征数据汇总和指标生命周期管理 可帮助您明智随着时间的推移管理您的数据。As with any enterprise system, you need tools to secure, manage, and monitor your Elasticsearch clusters. Security, monitoring, and administrative features that are integrated into Elasticsearch enable you to use Kibana as a control center for managing a cluster. Features like data rollups and index lifecycle management help you intelligently manage your data over time.


文章转载自产品与码农,如果涉嫌侵权,请发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

评论