Paper 2023/1441

Out of the Box Testing

Hubert Kario, Red Hat

In this paper we analyse typical timing data that can be collected over loopback interface, in local, and in metropolitan area networks. We evaluate performance of few statistical test for detecting differences in timing of server responses. The evaluated tests include the popular Box test, as well as sign test, Wilcoxon signed-rank test, and paired sample t-test. We found that the Box test offers poor performance, as it's an incorrect test to use for the measurements we collected. Use of appropriate tests also allows for robust differentiation between much smaller differences than the existing literature would suggest. We were able to detect side channels of single-digit CPU cycles over regular gigabit Ethernet. Those alternative tests were also found to be robust against noise in production networks, allowing detection of side channel of just few nanoseconds with 6 network hops between test systems.

Available format(s)
Attacks and cryptanalysis
Publication info
non-parametric testsside-channel attackstiming attacksnetwork attacksbox testsign test
Contact author(s)
hkario @ redhat com
2023-09-24: approved
2023-09-21: received
See all versions
Short URL
Creative Commons Attribution-ShareAlike


      author = {Hubert Kario},
      title = {Out of the Box Testing},
      howpublished = {Cryptology ePrint Archive, Paper 2023/1441},
      year = {2023},
      note = {\url{}},
      url = {}
Note: In order to protect the privacy of readers, does not use cookies or embedded third party content.