Paper 2022/784

Fully Privacy-Preserving Federated Representation Learning via Secure Embedding Aggregation

Jiaxiang Tang, Hong Kong University of Science and Technology
Jinbao Zhu, Hong Kong University of Science and Technology
Songze Li, Hong Kong University of Science and Technology
Kai Zhang, Lehigh University
Lichao Sun, Lehigh University
Abstract

We consider a federated representation learning framework, where with the assistance of a central server, a group of $N$ distributed clients train collaboratively over their private data, for the representations (or embeddings) of a set of entities (e.g., users in a social network). Under this framework, for the key step of aggregating local embeddings trained at the clients in a private manner, we develop a secure embedding aggregation protocol named SecEA, which provides information-theoretical privacy guarantees for the set of entities and the corresponding embeddings at each client $simultaneously$, against a curious server and up to $T < N/2$ colluding clients. As the first step of SecEA, the federated learning system performs a private entity union, for each client to learn all the entities in the system without knowing which entities belong to which clients. In each aggregation round, the local embeddings are secretly shared among the clients using Lagrange interpolation, and then each client constructs coded queries to retrieve the aggregated embeddings for the intended entities. We perform comprehensive experiments on various representation learning tasks to evaluate the utility and efficiency of SecEA, and empirically demonstrate that compared with embedding aggregation protocols without (or with weaker) privacy guarantees, SecEA incurs negligible performance loss (within 5%); and the additional computation latency of SecEA diminishes for training deeper models on larger datasets.

Metadata
Available format(s)
PDF
Category
Cryptographic protocols
Publication info
Preprint.
Keywords
federated learning federated representation learning private set union secure embedding aggregation
Contact author(s)
jathon tang @ gmail com
songzeli @ ust hk
History
2022-06-20: approved
2022-06-18: received
See all versions
Short URL
https://ia.cr/2022/784
License
Creative Commons Attribution-NonCommercial-NoDerivs
CC BY-NC-ND

BibTeX

@misc{cryptoeprint:2022/784,
      author = {Jiaxiang Tang and Jinbao Zhu and Songze Li and Kai Zhang and Lichao Sun},
      title = {Fully Privacy-Preserving Federated Representation Learning via Secure Embedding Aggregation},
      howpublished = {Cryptology ePrint Archive, Paper 2022/784},
      year = {2022},
      note = {\url{https://eprint.iacr.org/2022/784}},
      url = {https://eprint.iacr.org/2022/784}
}
Note: In order to protect the privacy of readers, eprint.iacr.org does not use cookies or embedded third party content.