A Quantitative Evaluation of Constant-Time Analysis Tools

Zhenzhi Lai; Zhiyuan Zhang; Tian Qiu; Van-Thuan Pham; Udaya Parampalli

Paper 2026/2682

A Quantitative Evaluation of Constant-Time Analysis Tools

Zhenzhi Lai

, University of Melbourne

Zhiyuan Zhang

, Max Planck Institute for Security and Privacy

Tian Qiu

, University of Melbourne

Van-Thuan Pham

, University of Melbourne

Udaya Parampalli

, University of Melbourne

Abstract

The constant-time programming policy protects cryptographic code from timing side-channels. However, writing constant-time code remains challenging even for experienced developers. Therefore, over 50 CT analysis tools have been published to help them identify constant-time violations. While several studies have examined those tools, they predominantly rely on qualitative observations and case studies, rather than quantitative assessments. Therefore, we present AutoBench-CT, a modular benchmark that manually verified per-violation ground truths across 64 cryptographic tasks and the first unified quantitative metrics to quantify the Operational Performance, Detection Quality, and Usability for seven such tools. With the results, we quantitatively show why CT-grind, the first tool published in 2010, is still the most popular tool in the wild. Based on the quantitative findings, we further provide new insights into recurring limitations across tools and propose a composite pipeline to maximize the security guarantees of constant-timeness in practice.

Metadata

Available format(s): PDF
Category: Implementation
Publication info: Published elsewhere. Minor revision. ACSAC 2026
Keywords: Constant-time analysis; Timing attacks.
Contact author(s): zhenzhil @ student unimelb edu au
zhiyuan zhang @ mpi-sp org
tq @ student unimelb edu au
thuan pham @ unimelb edu au
udaya @ unimelb edu au
History: 2026-10-31: approved; 2026-10-29: received; See all versions
Short URL: https://ia.cr/2026/2682
License: CC BY

BibTeX

@misc{cryptoeprint:2026/2682,
      author = {Zhenzhi Lai and Zhiyuan Zhang and Tian Qiu and Van-Thuan Pham and Udaya Parampalli},
      title = {A Quantitative Evaluation of Constant-Time Analysis Tools},
      howpublished = {Cryptology {ePrint} Archive, Paper 2026/2682},
      year = {2026},
      url = {https://eprint.iacr.org/2026/2682}
}