Paper 2022/169

SP 800-22 and GM/T 0005-2012 Tests: Clearly Obsolete, Possibly Harmful

Markku-Juhani O. Saarinen


When it comes to cryptographic random number generation, poor understanding of the security requirements and ``mythical aura'' of black-box statistical testing frequently leads it to be used as a substitute for cryptanalysis. To make things worse, a seemingly standard document, NIST SP 800-22, describes 15 statistical tests and suggests that they can be used to evaluate random and pseudorandom number generators in cryptographic applications. The Chinese standard GM/T 0005-2012 describes similar tests. These documents have not aged well. The weakest pseudorandom number generators will easily pass these tests, promoting false confidence in insecure systems. We strongly suggest that SP 800-22 be withdrawn by NIST; we consider it to be not just irrelevant but actively harmful. We illustrate this by discussing the ``reference generators'' contained in the SP 800-22 document itself. None of these generators are suitable for modern cryptography, yet they pass the tests. For future development, we suggest focusing on stochastic modeling of entropy sources instead of model-free statistical tests. Random bit generators should also be reviewed for potential asymmetric backdoors via trapdoor one-way functions, and for security against quantum computing attacks.

Note: Update History: 2022-Apr-12: Accepted to SSR 2022, updated. ( ) 2022-Mar-05: Expanded with additional commentary. 2022-Feb-15: Written as a Comment on "SP 800-22 Rev. 1a Decision Proposal." ( )

Available format(s)
Publication info
Published elsewhere. Minor revision.SSR 2022: The 7th Conference on Security Standards Research. June 6, 2022, Genoa, Italy.
TRNGEntropy SourcesSP 800-22GMT 0005-2012Statistical Randomness TestsStochastic Models
Contact author(s)
mjos @ mjos fi
2022-04-12: last of 7 revisions
2022-02-20: received
See all versions
Short URL
Creative Commons Attribution


      author = {Markku-Juhani O.  Saarinen},
      title = {SP 800-22 and GM/T 0005-2012 Tests: Clearly Obsolete, Possibly Harmful},
      howpublished = {Cryptology ePrint Archive, Paper 2022/169},
      year = {2022},
      note = {\url{}},
      url = {}
Note: In order to protect the privacy of readers, does not use cookies or embedded third party content.