Paper 2021/1688

Low-Complexity Deep Convolutional Neural Networks on Fully Homomorphic Encryption Using Multiplexed Parallel Convolutions

Eunsang Lee, Joon-Woo Lee, Junghyun Lee, Young-Sik Kim, Yongjune Kim, Jong-Seon No, and Woosuk Choi


Recently, the standard ResNet-20 network was successfully implemented on residue number system variant Cheon-Kim-Kim-Song (RNS-CKKS) scheme using bootstrapping, but the implementation lacks practicality due to high latency and low security level. To improve the performance, we first minimize total bootstrapping runtime using multiplexed parallel convolution that collects sparse output data for multiple channels compactly. We also propose the \emph{imaginary-removing bootstrapping} to prevent the deep neural networks from catastrophic divergence during approximate ReLU operations. In addition, we optimize level consumptions and use lighter and tighter parameters. Simulation results show that we have 4.67$\times$ lower inference latency and 134$\times$ less amortized runtime (runtime per image) for ResNet-20 compared to the state-of-the-art previous work, and we achieve standard 128-bit security. Furthermore, we successfully implement ResNet-110 with high accuracy on the RNS-CKKS scheme for the first time.

Available format(s)
Publication info
Published elsewhere. MINOR revision.ICML 2022
Artificial intelligenceCheon-Kim-Kim-Song (CKKS)ConvolutionFully homomorphic encryption (FHE)Privacy-preserving machine learning (PPML)ResNet model
Contact author(s)
eslee3209 @ ccl snu ac kr
shaeunsang @ snu ac kr
jsno @ snu ac kr
iamyskim @ chosun ac kr
yjk @ dgist ac kr
joonwoo42 @ snu ac kr
jhlee @ ccl snu ac kr
woosuk0 choi @ samsung com
2022-05-23: last of 3 revisions
2021-12-30: received
See all versions
Short URL
Creative Commons Attribution


      author = {Eunsang Lee and Joon-Woo Lee and Junghyun Lee and Young-Sik Kim and Yongjune Kim and Jong-Seon No and Woosuk Choi},
      title = {Low-Complexity Deep Convolutional Neural Networks on Fully Homomorphic Encryption Using Multiplexed Parallel Convolutions},
      howpublished = {Cryptology ePrint Archive, Paper 2021/1688},
      year = {2021},
      note = {\url{}},
      url = {}
Note: In order to protect the privacy of readers, does not use cookies or embedded third party content.