Neural Network Architecture Optimization through Submodularity and Supermodularity.

Deep learning modelsu0027 architectures, including depth and width, are key factors influencing modelsu0027 performance, such as test accuracy and computation time. This paper solves two problems: given computation time budget, choose an architecture to maximize accuracy, and given accuracy requirement, choose an architecture to minimize computation time. We convert this architecture optimization into a subset selection problem. With accuracyu0027s submodularity and computation timeu0027s supermodularity, we propose efficient greedy optimization algorithms. The experiments demonstrate our algorithmu0027s ability to find more accurate models or faster models. By analyzing architecture evolution with growing time budget, we discuss relationships among accuracy, time and architecture, and give suggestions on neural network architecture design.