Cryptology ePrint Archive: Report 2021/835

Practical, Label Private Deep Learning Training based on Secure Multiparty Computation and Differential Privacy

Sen Yuan and Milan Shen and Ilya Mironov and Anderson C. A. Nascimento

Abstract: Secure Multiparty Computation (MPC) is an invaluable tool for training machine learning models when the training data cannot be directly accessed by the model trainer. Unfortunately, complex algorithms, such as deep learning models, have their computational complexities increased by orders of magnitude when performed using MPC protocols. In this contribution, we study how to efficiently train an important class of machine learning problems by using MPC where features are known by one of the computing parties and only the labels are private. We propose new protocols combining differential privacy (DP) and MPC in order to privately and efficiently train a deep learning model in such scenario. More specifically, we release differentially private information during the MPC computation to dramatically reduce the training time. All released information does not compromise the privacy of the labels at the individual level. Our protocols can have running times that are orders of magnitude better than a straightforward use of MPC at a moderate cost in model accuracy.

Category / Keywords: cryptographic protocols / MPC, Differential Privacy, Privacy Preserving Machine Learning, DP-SGD

Date: received 18 Jun 2021, last revised 29 Jun 2021

Contact author: andclay at uw edu

Available format(s): PDF | BibTeX Citation

Version: 20210629:184733 (All versions of this report)

Short URL: ia.cr/2021/835


[ Cryptology ePrint archive ]