Paper 2026/174

STIP: Efficient and Secure Non-Interactive Transformer Inference via Compact Packing

Abstract

Secure TransFormer Inference (STFI) for LLMs aims to protect both user inputs and model parameters. Fully Homomorphic Encryption (FHE) offers a promising approach for STFI due to its non-interactivity, which eliminates communication overhead. However, FHE-based STFI incurs significant computational costs compared to plaintext inference. Recent advancements have accelerated inference by optimizing packing strategies and reducing the number of rotations. Despite these improvements, several challenges persist, including excessive rotations in ciphertext-ciphertext matrix multiplications (CCMMs), low input/output projection throughput, and expensive maximum/inverse operations, as well as wasted storage slots and inflated ciphertext counts due to sparse packing. To address these issues, we propose STIP, an efficient and secure non-interactive transformer inference framework that incorporates three novel packing strategies: (1) Real-Imaginary Hybrid Packing (RIHP) halves the rotation costs of CCMMs by enabling the simultaneous computation of two output results within the real and imaginary components; (2) Dual-Head Packing (DHP) maps adjacent heads to the real and imaginary components, doubling the throughput of attention projections; and (3) Adaptive Multi-Column Packing (AMCP) packs multiple heads into a single ciphertext, maximizing slot occupancy to reduce the total ciphertext count and thereby enhance computational parallelism. Moreover, for non-linear layers, we employ the Gaussian Kernel instead of Softmax, eliminating the need for maximum value searches and inverse operations, supported by a column-packed RIHP-based L2-norm algorithm. We reformulate LayerNorm into an inverse-free form by exploiting scale-invariance. Experimental results on a GPU show that STIP achieves approximately 1.6× speedup over the SOTA scheme Euston (S&P '26) on BERT-base, LLAMA-3-8B, and GPT-2-1.5B.