Paper 2026/1442

CoSecRAG: Efficient Vector Similarity Search with Database and Query Privacy for RAG

Abstract

Retrieval-augmented generation (RAG) grounds large language models on external knowledge by using vector similarity search over document embeddings to identify relevant documents and supplying their contents as generation context. However, its retrieval stage creates privacy risks: user queries may reveal sensitive intents, while the database may contain proprietary or confidential records. Existing secret-sharing-based solutions that protect both query and database privacy remain costly. To make private vector similarity search more efficient, we present CoSecRAG, a two-server system for RAG that protects both database and query embeddings. CoSecRAG introduces Inner Product Preserving Query Mask (IPQ-Mask) to lower similarity-scoring costs by avoiding online secure multiplications and Post-Scoring Cluster Pruning (PSCP) to reorganize the execution pipeline for approximate retrieval, reducing private-extraction volume and secure top-K selection costs. We implement CoSecRAG and evaluate it on six BEIR datasets and large-scale synthetic benchmarks. CoSecRAG maintains strong retrieval quality while substantially reducing the cost of private vector similarity search. Compared with P²RAG, the most recent two-server baseline for private vector similarity search in RAG, CoSecRAG achieves up to 181× online score-computation speedup and 81× online speedup in secure top-K computation. End-to-end, it achieves up to 99× online and 34× offline runtime speedups. Compared with PRAG, CoSecRAG achieves 19×–58× online and 2×–16× offline end-to-end speedups.