Paper 2023/1859

XorSHAP: Privacy-Preserving Explainable AI for Decision Tree Models

Dimitar Jetchev, Inpher Sarl
Marius Vuille, Inpher Sarl

Explainable AI (XAI) refers to the development of AI systems and machine learning models in a way that humans can understand, interpret and trust the predictions, decisions and outputs of these models. A common approach to explainability is feature importance, that is, determining which input features of the model have the most significant impact on the model prediction. Two major techniques for computing feature importance are LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations). While very generic, these methods are computationally expensive even in plaintext. Applying them in the privacy-preserving setting when part or all of the input data is private is therefore a major computational challenge. In this paper, we present $\texttt{XorSHAP}$ - the first practical privacy-preserving algorithm for computing Shapley values for decision tree ensemble models in the semi-honest Secure Multiparty Computation (SMPC) setting with full threshold. Our algorithm has complexity $O(T \widetilde{M} D 2^D)$, where $T$ is the number of decision trees in the ensemble, $D$ is the depth of the decision trees and $\widetilde{M}$ is the maximum of the number of features $M$ and $2^D$ (the number of leaf nodes of a tree), and scales to real-world datasets. Our implementation is based on Inpher's $\texttt{Manticore}$ framework and simultaneously computes (in the SMPC setting) the Shapley values for 100 samples for an ensemble of $T = 60$ trees of depth $D = 4$ and $M = 100$ features in just 7.5 minutes, meaning that the Shapley values for a single prediction are computed in just 4.5 seconds for the same decision tree ensemble model. Additionally, it is parallelization-friendly, thus, enabling future work on massive hardware acceleration with GPUs.

Available format(s)
Publication info
Explainable AIModel ExplainabilityGradient Boosting Decision TreesSHAP valuesSecure Multiparty Computation
Contact author(s)
dimitar @ inpher io
marius @ inpher io
2023-12-06: approved
2023-12-04: received
See all versions
Short URL
Creative Commons Attribution


      author = {Dimitar Jetchev and Marius Vuille},
      title = {XorSHAP: Privacy-Preserving Explainable AI for Decision Tree Models},
      howpublished = {Cryptology ePrint Archive, Paper 2023/1859},
      year = {2023},
      note = {\url{}},
      url = {}
Note: In order to protect the privacy of readers, does not use cookies or embedded third party content.