AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We present a comprehensive analysis of important metrics in practical applications: accuracy, memory footprint, parameters, operations count, inference time and power consumption

An Analysis of Deep Neural Network Models for Practical Applications.

arXiv: Computer Vision and Pattern Recognition, (2017)

Cited: 1031|Views347
EI
Full Text
Bibtex
Weibo

Abstract

Since the emergence of Deep Neural Networks (DNNs) as a prominent technique in the field of computer vision, the ImageNet classification challenge has played a major role in advancing the state-of-the-art. While accuracy figures have steadily increased, the resource utilisation of winning models has not been properly taken into account. I...More

Code:

Data:

0
Introduction
  • Since the emergence of Deep Neural Networks (DNNs) as a prominent technique in the field of computer vision, the ImageNet classification challenge has played a major role in advancing the state-of-the-art.
  • The authors present a comprehensive analysis of important metrics in practical applications: accuracy, memory footprint, parameters, operations count, inference time and power consumption.
Highlights
  • Since the emergence of Deep Neural Networks (DNNs) as a prominent technique in the field of computer vision, the ImageNet classification challenge has played a major role in advancing the state-of-the-art
  • We present a comprehensive analysis of important metrics in practical applications: accuracy, memory footprint, parameters, operations count, inference time and power consumption
  • Key findings are: (1) power consumption is independent of batch size and architecture; (2) accuracy and inference time are in a hyperbolic relationship; (3) energy constraint are an upper bound on the maximum achievable accuracy and model complexity; (4) the number of operations is a reliable estimate of the inference time
  • In this paper we analysed multiple state-of-the-art deep neural networks submitted to the ImageNet challenge, in terms of accuracy, memory footprint, parameters, operations count, inference time and power consumption
  • We show that accuracy and inference time are in a hyperbolic relationship: a little increment in accuracy costs a lot of computational time
  • We show that an energy constraint will set a specific upper bound on the maximum achievable accuracy and model complexity, in terms of operations counts
Results
  • Key findings are: (1) power consumption is independent of batch size and architecture; (2) accuracy and inference time are in a hyperbolic relationship; (3) energy constraint are an upper bound on the maximum achievable accuracy and model complexity; (4) the number of operations is a reliable estimate of the inference time.
  • The authors compare these architectures on multiple metrics related to resource utilisation in actual deployments: accuracy, memory footprint, parameters, operations count, inference time and power consumption.
  • The authors analysed the following DDNs: AlexNet (Krizhevsky et al, 2012), batch normalised AlexNet (Zagoruyko, 2016), batch normalised Network In Network (NIN) (Lin et al, 2013), ENet (Paszke et al, 2016) for ImageNet (Culurciello, 2016), GoogLeNet (Szegedy et al, 2014), VGG-16 and -19 (Simonyan & Zisserman, 2014), ResNet-18,
  • Figure 1 shows one-crop accuracies of the most relevant entries submitted to the ImageNet challenge, from the AlexNet (Krizhevsky et al, 2012), on the far left, to the best performing Inception-v4 (Szegedy et al, 2016).
  • The authors analyse the relationship between power consumption and number of operations required by a given model.
  • Given that the operations count is linear with the inference time, the authors get that the accuracy has an hyperbolical dependency on the amount of computations that a network requires.
  • ENet (Paszke et al, 2016) — which the authors have designed to be highly efficient and it has been adapted and retrained on ImageNet (Culurciello, 2016) for this work — achieves the highest score, showing that 24× less parameters are sufficient to provide state-of-the-art results.
  • In this paper the authors analysed multiple state-of-the-art deep neural networks submitted to the ImageNet challenge, in terms of accuracy, memory footprint, parameters, operations count, inference time and power consumption.
Conclusion
  • The authors' goal is to provide insights into the design choices that can lead to efficient neural networks for practical application, and optimisation of the often-limited resources in actual deployments, which lead them to the creation of ENet — or Efficient-Network — for ImageNet. The authors show that accuracy and inference time are in a hyperbolic relationship: a little increment in accuracy costs a lot of computational time.
  • The authors show that an energy constraint will set a specific upper bound on the maximum achievable accuracy and model complexity, in terms of operations counts.
Funding
  • This work is partly supported by the Office of Naval Research (ONR) grants N00014-12-10167, N00014-15-1-2791 and MURI N00014-10-1-0278
  • We gratefully acknowledge the support of NVIDIA Corporation with the donation of the TX1, Titan X, K40 GPUs used for this research
Reference
  • Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. cuDNN: Efficient Primitives for Deep Learning. arXiv.org arXiv:1410.0759, 2014.
    Findings
  • Ronan Collobert, Koray Kavukcuoglu, and Clement Farabet. Torch7: A matlab-like environment for machine learning. In BigLearn, NIPS Workshop, number EPFL-CONF-192376, 2011.
    Google ScholarLocate open access versionFindings
  • Eugenio Culurciello. Training enet. https://culurciello.github.io/tech/2016/06/20/training-enet.html, 2016.
    Findings
  • Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015.
    Findings
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385, 2015.
    Findings
  • Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105, 2012.
    Google ScholarLocate open access versionFindings
0
Your rating :

No Ratings

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn