Paper 2025/2318
Hyperion: Private Token Sampling with Homomorphic Encryption
, University of California, Santa Barbara, University of California, Santa Barbara, University of California, Santa Barbara, University of California, Santa Barbara, University of California, Santa BarbaraAbstract
A promising direction for enabling private queries to large language models (LLMs) is with homomorphic encryption (HE). An open problem is performing token sampling under HE. In this paper, we introduce Hyperion, an efficient HE algorithm for inverse transform sampling, enabling private token sampling with 1 comparison depth, $O(1)$ amortized comparisons, and $O(\log n)$ rotations. We implement our approach and demonstrate that it samples tokens in 0.14 seconds for 32k tokens ($\approx 4.4\, \mu\mathrm{s}$ per token) on GPU, achieving a $100\times$ latency improvement over prior work.
Metadata
- Available format(s)
-
PDF
- Category
- Applications
- Publication info
- Published elsewhere. Minor revision. ACL 2026
- DOI
- 10.18653/v1/2026.acl-long.644
- Keywords
- Private Token SamplingPrivate LLM InferenceHomomorphic Encryption
- Contact author(s)
-
lawrenceklim @ ucsb edu
jiamingliu @ ucsb edu
vikaskalagi @ ucsb edu
divyagrawal @ ucsb edu
elabbadi @ ucsb edu - History
- 2026-07-03: last of 5 revisions
- 2025-12-23: received
- See all versions
- Short URL
- https://ia.cr/2025/2318
- License
-
CC BY
BibTeX
@misc{cryptoeprint:2025/2318,
author = {Lawrence Lim and Jiaming Liu and Vikas Kalagi and Divyakant Agrawal and Amr El Abbadi},
title = {Hyperion: Private Token Sampling with Homomorphic Encryption},
howpublished = {Cryptology {ePrint} Archive, Paper 2025/2318},
year = {2025},
doi = {10.18653/v1/2026.acl-long.644},
url = {https://eprint.iacr.org/2025/2318}
}
