Paper 2023/1174

zkDL: Efficient Zero-Knowledge Proofs of Deep Learning Training

Haochen Sun, University of Waterloo
Tonghe Bai, University of Waterloo
Jason Li, University of Waterloo
Hongyang Zhang, University of Waterloo
Abstract

The recent advancements in deep learning have brought about significant changes in various aspects of people's lives. Meanwhile, these rapid developments have raised concerns about the legitimacy of the training process of deep neural networks. To protect the intellectual properties of AI developers, directly examining the training process by accessing the model parameters and training data is often prohibited for verifiers. In response to this challenge, we present zero-knowledge deep learning (zkDL), an efficient zero-knowledge proof for deep learning training. To address the long-standing challenge of verifiable computations of non-linearities in deep learning training, we introduce zkReLU, a specialized proof for the ReLU activation and its backpropagation. zkReLU turns the disadvantage of non-arithmetic relations into an advantage, leading to the creation of FAC4DNN, our specialized arithmetic circuit design for modelling neural networks. This design aggregates the proofs over different layers and training steps, without being constrained by their sequential order in the training process. With our new CUDA implementation that achieves full compatibility with the tensor structures and the aggregated proof design, zkDL enables the generation of complete and sound proofs in less than a second per batch update for an 8-layer neural network with 10M parameters and a batch size of 64, while provably ensuring the privacy of data and model parameters. To our best knowledge, we are not aware of any existing work on zero-knowledge proof of deep learning training that is scalable to million-size networks.

Metadata
Available format(s)
PDF
Category
Applications
Publication info
Preprint.
Keywords
Machine learningZero knowledge proofs
Contact author(s)
haochen sun @ uwaterloo ca
t4bai @ uwaterloo ca
j2643li @ uwaterloo ca
hongyang zhang @ uwaterloo ca
History
2023-12-08: revised
2023-07-30: received
See all versions
Short URL
https://ia.cr/2023/1174
License
Creative Commons Attribution
CC BY

BibTeX

@misc{cryptoeprint:2023/1174,
      author = {Haochen Sun and Tonghe Bai and Jason Li and Hongyang Zhang},
      title = {{zkDL}: Efficient Zero-Knowledge Proofs of Deep Learning Training},
      howpublished = {Cryptology {ePrint} Archive, Paper 2023/1174},
      year = {2023},
      url = {https://eprint.iacr.org/2023/1174}
}
Note: In order to protect the privacy of readers, eprint.iacr.org does not use cookies or embedded third party content.