Bagua！并行通讯库.pdf
50墨值下载
A Modular & Efficient Communication
Layer for Distributed Learning
Xiangru Lian, Rui Wang, Hongmei Shi,
Shengzhuo Zhang, Yan Tian, Ji Liu
Shaoduo Gan, Jiawei Jiang,
Ce Zhang
Deep Learning Eating the World
▪ Fascinating progress over the last decade,
fueled by increasingly large dataset and
computation power.
▪ The availability of computation changes our
view on what we can do.
▪ What used to take weeks now take hours
(Think about ImageNet); but users get
“greedy” with new tasks that take weeks on
a beefy machine. (Think about BERT)
At Kuaishou:
▪ Recommendation system
(300+M daily users' behaviors)
▪ Multimedia understanding
(dozens of thousands years
daily accumulated watching
time )
▪ Video preprocessing (~200
videos are uploaded per sec)
EC2 Instance: g2.8xlarge
• 4x GRID K520, 4.89 TFLOPS
• $2.60 / hour
EC2 Instance: p4d.24xlarge
• 8x A100, ~ 2.5 PFLOPS
• $ 32.77/h
of 30
50墨值下载
【版权声明】本文为墨天轮用户原创内容，转载时必须标注文档的来源（墨天轮），文档链接，文档作者等基本信息，否则作者和墨天轮有权追究责任。如果您发现墨天轮中有涉嫌抄袭或者侵权的内容，欢迎发送邮件至：contact@modb.pro进行举报，并提供相关证据，一经查实，墨天轮将立刻删除相关内容。
下载排行榜
文档被以下合辑收录
评论