Paper 2024/1870

A Divide-and-Conquer Strategy for Hard-Label Extraction of Deep Neural Networks via Side-Channel Attacks

Abstract

During the past decade, Deep Neural Networks (DNNs) have proven their value across a wide variety of applications; however, despite their importance, protecting their intellectual property remains an open issue. Recent work has successfully extracted DNNs using cryptanalytic methods in hard-label settings, showing that it is possible to copy a DNN with high fidelity, i.e., a high degree of similarity in correct/incorrect output predictions which corresponds to the proportion of samples for which the extracted model produces the same prediction as the original model. However, these methods have only been demonstrated on Multi-Layer Perceptrons (MLPs) and are sensitive to non–fully connected layers and special-case neurons. To overcome these limitations, we base our contribution on a divide-and-conquer paradigm. We introduce a new black-box side-channel attack that splits the targeted DNN into several linear components, for which cryptanalytic extraction can be performed. Building on this decomposition, we propose an end-to-end framework specifically designed for hard-label settings, not limited to fully connected layers, and robust to special-case neurons, while improving extraction fidelity. We validate our contribution by successfully extracting all architectures previously targeted in the literature, as well as several new architectures implemented on a microcontroller unit. These include an MLP with $1.7$ million parameters, nearly doubling the previous largest number of extracted weights, and a shortened MobileNetv1, which for the first time includes pooling layers and depthwise separable convolutions. Our framework successfully extracts all of these DNNs with high fidelity ($88.4\%$ for MobileNetv1 and $93.2\%$ for the MLP). Finally, we use the copied model to generate adversarial examples and achieve near white-box performance on the victim model ($95.8\%$ and $96.7\%$ transfer rates).