Breaking the Spell: Cryptanalysis of VDOO

Max Cartor; Ryann Cartor; Ansta Rakotondrafara

Paper 2026/2178

Breaking the Spell: Cryptanalysis of VDOO

Max Cartor

, Bellarmine University

Ryann Cartor

, Clemson University

Ansta Rakotondrafara

, Clemson University

Abstract

VDOO is a Rainbow-like multivariate signature scheme with additional ``diagonal” structure intended to resist structural attacks. We show that its security is governed primarily by the first two oil layers, while the diagonal structure has negligible impact, and that all proposed parameters fail to meet their claimed NIST security levels. We provide a corrected complexity analysis of the Simple and Rectangular MinRank attacks, introduce a hybrid Simple–Padded MinRank attack, and compare their effectiveness. Open-source code is provided for independent validation, clarifying VDOO’s true security and informing future Rainbow-type post-quantum designs.

Metadata

Available format(s): PDF
Category: Attacks and cryptanalysis
Publication info: Preprint.
Keywords: VDOO Cryptanalysis UOV Family
Contact author(s): mcartor01 @ bellarmine edu
rcartor @ clemson edu
arakoto @ g clemson edu
History: 2026-09-26: approved; 2026-09-23: received; See all versions
Short URL: https://ia.cr/2026/2178
License: CC0

BibTeX

@misc{cryptoeprint:2026/2178,
      author = {Max Cartor and Ryann Cartor and Ansta Rakotondrafara},
      title = {Breaking the Spell: Cryptanalysis of {VDOO}},
      howpublished = {Cryptology {ePrint} Archive, Paper 2026/2178},
      year = {2026},
      url = {https://eprint.iacr.org/2026/2178}
}