Paper 2026/198
ELLMo: Packing- and Depth-Aware Encrypted Transformer Inference
, Boston University, Boston University, University of Murcia, Korea Advanced Institute of Science and Technology, Korea Advanced Institute of Science and Technology, University of Murcia, Northeastern University, Boston UniversityAbstract
Cloud-based Large Language Model (LLM) inference processes sensitive user inputs, yet current deployments offer limited confidentiality guarantees. Fully Homomorphic Encryption (FHE) can provide strong privacy, but it clashes with transformer architectures, where rigid ciphertext packing demands expensive rotations, and deep polynomial circuits for nonlinearities necessitate costly bootstrapping. Although recent work has reported promising speed-ups, maintaining model accuracy is a challenge. We address these issues with ELLMo, a packing- and depth-aware encrypted transformer design. ELLMo introduces a novel matrix multiplication algorithm to reduce the ciphertext rotations. Further, head-split and merge steps are fused into this new algorithm at no additional cost. To reduce the depth of nonlinear layers, our contributions, Statistical-max Softmax and DelayNorm, help bypass deep comparison trees and homomorphic divisions to reduce bootstrapping by up to 46%. On encrypted BERT-Tiny, ELLMo achieves a $1.4\times$ speedup over state-of-the-art baselines with 0-1.5% accuracy loss across SST-2, MRPC, and RTE downstream tasks.
Metadata
- Available format(s)
-
PDF
- Category
- Applications
- Publication info
- Preprint.
- Keywords
- Homomorphic EncryptionPrivacy-Preserving Transformer Inference
- Contact author(s)
- seyda @ bu edu
- History
- 2026-07-28: revised
- 2026-02-07: received
- See all versions
- Short URL
- https://ia.cr/2026/198
- License
-
CC BY
BibTeX
@misc{cryptoeprint:2026/198,
author = {Seyda Nur Guzelhan and Lohit Daksha and Carlos Agulló Domingo and Gilbert Jonatan and John Kim and Jose L. Abellan and David Kaeli and Ajay Joshi},
title = {{ELLMo}: Packing- and Depth-Aware Encrypted Transformer Inference},
howpublished = {Cryptology {ePrint} Archive, Paper 2026/198},
year = {2026},
url = {https://eprint.iacr.org/2026/198}
}
