Paper 2023/1174
zkDL: Efficient Zero-Knowledge Proofs of Deep Learning Training
Abstract
The recent advancements in deep learning have brought about significant changes in various aspects of people's lives. Meanwhile, these rapid developments have raised concerns about the legitimacy of the training process of deep neural networks. To protect the intellectual properties of AI developers, directly examining the training process by accessing the model parameters and training data is often prohibited for verifiers. In response to this challenge, we present zero-knowledge deep learning (zkDL), an efficient zero-knowledge proof for deep learning training. To address the long-standing challenge of verifiable computations of non-linearities in deep learning training, we introduce zkReLU, a specialized proof for the ReLU activation and its backpropagation. zkReLU turns the disadvantage of non-arithmetic relations into an advantage, leading to the creation of FAC4DNN, our specialized arithmetic circuit design for modelling neural networks. This design aggregates the proofs over different layers and training steps, without being constrained by their sequential order in the training process. With our new CUDA implementation that achieves full compatibility with the tensor structures and the aggregated proof design, zkDL enables the generation of complete and sound proofs in less than a second per batch update for an 8-layer neural network with 10M parameters and a batch size of 64, while provably ensuring the privacy of data and model parameters. To our best knowledge, we are not aware of any existing work on zero-knowledge proof of deep learning training that is scalable to million-size networks.
Metadata
- Available format(s)
- Category
- Applications
- Publication info
- Preprint.
- Keywords
- Machine learningZero knowledge proofs
- Contact author(s)
-
haochen sun @ uwaterloo ca
t4bai @ uwaterloo ca
j2643li @ uwaterloo ca
hongyang zhang @ uwaterloo ca - History
- 2023-12-08: revised
- 2023-07-30: received
- See all versions
- Short URL
- https://ia.cr/2023/1174
- License
-
CC BY
BibTeX
@misc{cryptoeprint:2023/1174, author = {Haochen Sun and Tonghe Bai and Jason Li and Hongyang Zhang}, title = {{zkDL}: Efficient Zero-Knowledge Proofs of Deep Learning Training}, howpublished = {Cryptology {ePrint} Archive, Paper 2023/1174}, year = {2023}, url = {https://eprint.iacr.org/2023/1174} }