Cryptology ePrint Archive: Report 2017/138

How (not) to Use Welch's T-test in Side-Channel Security Evaluations

François-Xavier Standaert

Abstract: The Test Vector Leakage Assessment (TVLA) methodology is a qualitative tool relying on Welch's T-test to assess the security of cryptographic implementations against side-channel attacks. Despite known limitations (e.g., risks of false negatives and positives), it is sometimes considered as a pass-fail test to determine whether such implementations are "safe" or not (without clear definition of what is "safe"). In this note, we clarify the limited quantitative meaning of this test when used as a standalone tool. For this purpose, we first show that the straightforward application of this approach to assess the security of a masked implementation is not sufficient. More precisely, we show that even in a simple (more precisely, univariate) case study that seems best suited for the TVLA methodology, detection (or lack thereof) with Welch's T-test can be totally disconnected from the actual security level of an implementation. For this purpose, we put forward the case of a realistic masking scheme that looks very safe from the TVLA point-of-view and is nevertheless easy to break. We then discuss this result in more general terms and argue that this limitation is shared by all "moment-based" security evaluations. We conclude the note positively, by describing how to use moment-based analyzes as a useful ingredient of side-channel security evaluations, to determine a "security order".

Category / Keywords: side-channel analysis, security evaluations

Original Publication (in the same form): Proceedings of CARDIS 2018

Date: received 15 Feb 2017, last revised 15 Oct 2018

Contact author: fstandae at uclouvain be

Available format(s): PDF | BibTeX Citation

Version: 20181015:081336 (All versions of this report)

Short URL:

[ Cryptology ePrint archive ]