xMLP: Revolutionizing Private Inference with Exclusive Square Activation
arxiv(2024)
摘要
Private Inference (PI) enables deep neural networks (DNNs) to work on private
data without leaking sensitive information by exploiting cryptographic
primitives such as multi-party computation (MPC) and homomorphic encryption
(HE). However, the use of non-linear activations such as ReLU in DNNs can lead
to impractically high PI latency in existing PI systems, as ReLU requires the
use of costly MPC computations, such as Garbled Circuits. Since square
activations can be processed by Beaver's triples hundreds of times faster
compared to ReLU, they are more friendly to PI tasks, but using them leads to a
notable drop in model accuracy. This paper starts by exploring the reason for
such an accuracy drop after using square activations, and concludes that this
is due to an "information compounding" effect. Leveraging this insight, we
propose xMLP, a novel DNN architecture that uses square activations exclusively
while maintaining parity in both accuracy and efficiency with ReLU-based DNNs.
Our experiments on CIFAR-100 and ImageNet show that xMLP models consistently
achieve better performance than ResNet models with fewer activation layers and
parameters while maintaining consistent performance with its ReLU-based
variants. Remarkably, when compared to state-of-the-art PI Models, xMLP
demonstrates superior performance, achieving a 0.58
7x faster PI speed. Moreover, it delivers a significant accuracy improvement of
4.96
xMLP is up to 700x faster than the previous state-of-the-art PI model with
comparable accuracy.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要