In this paper, we present new and efficient protocols for privacy preserving machine learning for linear regression, logistic regression and neural network training using the stochastic gradient descent method. Our protocols fall in the two-server model where data owners distribute their private data among two non-colluding servers who train various models on the joint data using secure two-party computation (2PC). We develop new techniques to support secure arithmetic operations on shared decimal numbers, and propose MPC-friendly alternatives to nonlinear functions such as sigmoid and softmax that are superior to prior work.
We implement our system in C++. Our experiments validate that our protocols are several orders of magnitude faster than the state of the art implementations for privacy preserving linear and logistic regressions, and scale to millions of data samples with thousands of features. We also implement the first privacy preserving system for training neural networks.
Category / Keywords: cryptographic protocols / Privacy-preserving machine learning, secure computation. Original Publication (with minor differences): IEEE Symposium on Security and Privacy 2017 Date: received 19 Apr 2017, last revised 7 Jun 2017 Contact author: zhangyp at umd edu Available format(s): PDF | BibTeX Citation Note: Fix a typo for equations in Section 3 and 4. Version: 20170607:175403 (All versions of this report) Short URL: ia.cr/2017/396 Discussion forum: Show discussion | Start new discussion