ApproxCaliper: A Programmable Framework for Application-aware Neural Network Optimization

To deploy compute-intensive neural networks on resource-constrained edge systems, developers use model optimization techniques that reduce model size and computational cost. Existing optimization tools are application-agnostic – they optimize model parameters solely in view of the neural network accuracy – and can thus miss optimization opportunities. We propose ApproxCaliper , the first programmable framework for application-aware neural network optimization . By incorporating application-specific goals, ApproxCaliper facilitates more aggressive optimization of the neural networks compared to application-agnostic techniques. We perform experiments on five different neural networks used in two real-world robotics systems: a commercial agriculture robot and a simulation of an autonomous electric cart. Compared to Learning Rate Rewinding (LRR), a state-of-the-art structured pruning tool used in an application agnostic setting, ApproxCaliper achieves 5.3 × higher speedup and 2.9 × lower GPU resource utilization, and 36 × and 6.1 × additional model size reduction for the two evaluated benchmarks, respectively.