Hadoop_on_Ampere_Arm_Processors_Ref_Architecture_0_75_20231024_f5784a93f6.pdf
5墨值下载
Hadoop on Ampere® Arm Processors Reference Architecture
Ampere Computing Proprietary
AMP 2023-0057
Hadoop on Ampere® Arm Processors
Reference Architecture
October 24, 2023
Document Issue 0.75
Hadoop on Ampere® Arm Processors Reference Architecture
Document Issue 0.75
Ampere Computing Proprietary
2
Contents
1. Overview ...................................................................................................................................................................................... 4
1.1 Scope and Audience ............................................................................................................................................................ 4
1.2 Ampere Altra Max Processors ............................................................................................................................................ 4
2. Big Data Architecture .................................................................................................................................................................. 5
2.1 Components ........................................................................................................................................................................ 5
2.2 Hadoop Ecosystem .............................................................................................................................................................. 6
2.2.1 Hadoop Distributed File System (HDFS)..................................................................................................................... 6
2.2.2 MapReduce ................................................................................................................................................................. 6
2.2.3 Yet Another Resource Negotiator (YARN) ................................................................................................................. 6
2.2.4 Hadoop Common ........................................................................................................................................................ 6
2.3 Hadoop Test Bed ................................................................................................................................................................. 7
2.3.1 Equipment Under Test ................................................................................................................................................ 7
3. Hadoop Installation and Cluster Setup ....................................................................................................................................... 8
3.1 Configure Unified Extensible Firmware Interface (UEFI)................................................................................................... 8
3.2 Install the OS ........................................................................................................................................................................ 8
3.2.1 Set Up Networking ...................................................................................................................................................... 8
3.2.2 Set Up Storage ............................................................................................................................................................. 8
3.2.3 Create the Hadoop User ............................................................................................................................................. 8
3.2.4 After the OS Install ...................................................................................................................................................... 8
3.3 Install Hadoop ...................................................................................................................................................................... 8
3.4 Verify the Installation .......................................................................................................................................................... 9
4. Performance Tuning .................................................................................................................................................................. 10
4.1 UEFI .................................................................................................................................................................................... 10
4.2 Linux ................................................................................................................................................................................... 10
4.3 Network ............................................................................................................................................................................. 10
4.4 Disks ................................................................................................................................................................................... 10
4.5 HDFS, YARN and MapReduce ............................................................................................................................................ 10
5. Benchmarking Tools .................................................................................................................................................................. 11
6. Performance Tests on Three-Node Clusters ............................................................................................................................ 12
6.1 TeraSort Performance ....................................................................................................................................................... 12
6.2 Wordcount Performance .................................................................................................................................................. 12
6.3 CPU Utilization ................................................................................................................................................................... 13
6.4 Disk and Network Utilization ............................................................................................................................................ 13
6.5 Power Consumption .......................................................................................................................................................... 14
6.6 Rack and Datacenter Level Efficiency ............................................................................................................................... 14
of 26
5墨值下载
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文档的来源（墨天轮），文档链接，文档作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。
下载排行榜
评论