Error Analysis of NIST SP 800-22 Test Suite

Statistical tests for randomness play an essential role in cryptography, but the reliability of these tests is rarely taken into account, which may mislead our judgement of randomness especially with extremely large samples. In this paper, we make a two-stage error analysis of the commonly used two-level randomness tests in the NIST SP 800–22 test suite. Especially, we give the estimates of the ${p}$ -value deviations of chi-square approximation in the basic tests based on our proposed continuity constraints, and mathematically express the reliability of uniformity test used in the test suite with the fact of noncentral chi-square approximation. Finally, we analyze the respective error factors and derive the corresponding probability deviation estimation for the tests, explaining some false positive issues in practical test experiments. With our derived error analysis models, one can get a more reliable randomness test strategy with extremely large samples and wider parameter selections.