Cryptology ePrint Archive: Report 2018/442

SecureNN: Efficient and Private Neural Network Training

Sameer Wagh and Divya Gupta and Nishanth Chandran

Abstract: Neural Networks (NN) provide a powerful method for machine learning training and prediction. For effective training, it is often desirable for multiple parties to combine their data -- however, doing so conflicts with data privacy. In this work, we provide novel three-party and four-party secure computation protocols for various NN building blocks such as matrix multiplication, Rectified Linear Units, MaxPool, normalization etc. This enables us to construct three-party and four-party information-theoretically secure protocols for training and prediction of CNNs, DNNs and a number of other NN architectures such that no single party learns any information about the data.

Experimentally, we build a system and train a (A) 3-layer DNN (B) 4-layer CNN from MiniONN, and (C) 4-layer LeNet network. Compared to the state-of-the-art prior work SecureML (Mohassel and Zhang, IEEE S&P 2017) that provided (computationally-secure) protocols for only the network A in the 2 and 3-party setting, we obtain 93X and 8X improvements, respectively. In the WAN setting, these improvements are more drastic - for example, we obtain an improvement of 407X. Our efficiency gains come from a >8X improvement in communication, coupled with the complete elimination of expensive oblivious transfer protocols. In fact, our results show that the overhead of executing secure training protocols is only between 17-33X of the cleartext implementation even for networks that achieve >99% accuracy.

Category / Keywords: cryptographic protocols / secure computation, neural network training, information-theoretic security

Date: received 10 May 2018, last revised 14 May 2018

Contact author: nichandr at microsoft com, t-digu@microsoft com, snwagh@gmail com

Available format(s): PDF | BibTeX Citation

Version: 20180514:150605 (All versions of this report)

Short URL:

[ Cryptology ePrint archive ]