基本信息
浏览量:14
职业迁徙
个人简介
I make fast machine learning algorithms.
I do this because 1) I’m good at it, and 2) I claim that it’s extremely important. The reason it’s important is that speed helps us get all the other qualities we like in machine learning, such as accuracy, privacy, fairness, and safety. It does this in three ways:
It directly lets you get more of these qualities through usage of larger models, exploration of more architectures or hyperparameters, slower but privacy-preserving training, slower but adversarially robust inference, etc.
A corollary is that it improves incentives. The history of technology shows that desirable features take a back seat to essential features. In machine learning today, we’re seeing model accuracy and inference time dominate desirable traits like safety and privacy. By making it easy to obtain high-quality models in the essential respects, we make it worthwhile to consider the “non-essential” respects as well.
It facilitates research, especially in academia. The bottleneck in a great deal of machine learning research, particularly for those of us who lack tech-giant-level resources, is experiment time. Faster machine learning means faster research progress. The disproportionate aid to academics also (at least in theory) disproportionately facilitates research on socially desirable aspects of machine learning such as privacy, safety, and fairness that private companies may have less incentive to pursue.
There’s more specific reasoning behind my individual projects, but these points hopefully give you a taste of why I think speed is important.
I’ve had a lot of failures in pursuing this, but also some successes. My favorite successes so far include:
Learning to recognize spoken words from five unlabeled examples in under two seconds [1]
Training on data at 5GB/s in a single thread [2]
Multiplying matrices 10x faster than a matrix multiply (with some approximation error) [3]
Nearest-neighbor searching through billions of images per second in one thread with no indexing [3]
I do this because 1) I’m good at it, and 2) I claim that it’s extremely important. The reason it’s important is that speed helps us get all the other qualities we like in machine learning, such as accuracy, privacy, fairness, and safety. It does this in three ways:
It directly lets you get more of these qualities through usage of larger models, exploration of more architectures or hyperparameters, slower but privacy-preserving training, slower but adversarially robust inference, etc.
A corollary is that it improves incentives. The history of technology shows that desirable features take a back seat to essential features. In machine learning today, we’re seeing model accuracy and inference time dominate desirable traits like safety and privacy. By making it easy to obtain high-quality models in the essential respects, we make it worthwhile to consider the “non-essential” respects as well.
It facilitates research, especially in academia. The bottleneck in a great deal of machine learning research, particularly for those of us who lack tech-giant-level resources, is experiment time. Faster machine learning means faster research progress. The disproportionate aid to academics also (at least in theory) disproportionately facilitates research on socially desirable aspects of machine learning such as privacy, safety, and fairness that private companies may have less incentive to pursue.
There’s more specific reasoning behind my individual projects, but these points hopefully give you a taste of why I think speed is important.
I’ve had a lot of failures in pursuing this, but also some successes. My favorite successes so far include:
Learning to recognize spoken words from five unlabeled examples in under two seconds [1]
Training on data at 5GB/s in a single thread [2]
Multiplying matrices 10x faster than a matrix multiply (with some approximation error) [3]
Nearest-neighbor searching through billions of images per second in one thread with no indexing [3]
研究兴趣
论文共 11 篇作者统计合作学者相似作者
按年份排序按引用量排序主题筛选期刊级别筛选合作者筛选合作机构筛选
时间
引用量
主题
期刊级别
合作者
合作机构
arxiv(2021)
引用29浏览0EI引用
29
0
MLSys (2020): 129-146
引用591浏览0EI引用
591
0
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologiesno. 3 (2018): 1-23
user-5d8054e8530c708f9920ccce(2018)
引用2浏览0引用
2
0
CoRRno. 3 (2018): 1-23
加载更多
作者统计
合作学者
合作机构
D-Core
- 合作者
- 学生
- 导师
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn