Optimizations of Privacy-Preserving DNN for Low-Latency Inference on Encrypted Data.

Homomorphic encryption (HE) based on the CKKS scheme is a promising candidate for implementing privacy-preserving deep neural networks (PP-DNN) by performing operations directly on the encrypted data. However, due to the computational complexity of HE operation, even simple PP-DNNs require a huge amount of processing time. In order to reduce the processing time of PP-DNN, in this paper, we present an innovative, low-latency model optimization solution for PP-DNNs. Our proposed low-latency model optimization solution exploits second-order polynomials that approximate original activation functions, ensuring low-latency and accurate DNN performance. To further reduce the processing latency of PP-DNNs, we introduce the coefficient absorbing technique and a masking convolution for convolutional layers. The experimental results show that the proposed solution constructs bootstrapping-free PP-DNN and reduces the inference latency of CKKS-based ResNet-34 by 35% in the CIFAR-100 dataset and ResNet-32 by 77% in the CIFAR-10 dataset compared to previous approaches while maintaining the same level of inference accuracy. Moreover, through the layer-wise latency analysis, we show the efficacy of our approaches, and through validation in various scenarios, we demonstrate the generality of our methods.