A Study of Machine Learning Inference Benchmarks

Machine learning (ML) is becoming a powerful tool for a variety of applications where artificial intelligence solutions are required. A ML benchmark is a standard suite to measure, evaluate and compare the performance and efficiency of ML systems. This study analyzes the benchmark results from two famous benchmarks MLMark and MLPerf to provide a basis of comparison between both benchmarks as well as to provide recommendations on computer architectures to utilize for ML inferencing. Lastly, special emphasis is placed on the performance of edge AI devices.