Paper 2026/168

Cryptanalytic Extraction of Recurrent Neural Network Models

Abstract

In recent years, neural network extraction has been studied with cryptographic techniques, since Carlini et al.'s pioneering work proposed at CRYPTO 2020. Most research has focused on simple fully connected network (FCN) models, with limited attention given to more complicated recurrent neural network (RNN) models. However, RNN models are dominant in fields such as natural language processing and speech recognition. Exploring the vulnerability of RNN models to extraction attacks is not only methodologically significant but also reveals an attack surface broader in scope and higher in real-world impact. In this work, for the first time we propose a series of cryptanalytic extraction attacks against RNN models under both the raw-output (S5) and hard-label (S1) scenarios. Our attack selects inputs to establish an equivalence between the RNN and shallow FCN models. Since the parameters of these equivalent models are entangled with neuron permutations and scaling factors, they must be aligned before reuse. In the S5 scenario, we construct an equivalent FCN model and apply permutation and scaling alignment methods to enable parameter reuse. In the S1 scenario, we establish an equivalence between one RNN and two FCN models, and propose permutation search, accuracy enhancement and sign search methods to address the challenges of hard-label scenarios. In the S5 scenario, we recover the parameters of five RNN models with different configurations, while in the S1 scenario, we recover those of two RNN models, and in both cases the models reach depths of up to 1024 layers. To the best of our knowledge, this is the first time that model extraction attacks have been extended from networks with fewer than 10 layers to networks with thousands of layers. All experiments are completed on a PC within two hours.