QNGmeter Continuous Real-Time Statistical Tester

QNGmeter 3.6 Documentation

v2019.11.15 © 2019 The Quantum World Corporation.

Overview
The QNGmeter is a continuous statistical tester that uses five powerful and fundamentally different tests on the input data. This version tests ComScire QNG hardware as well as external sources through a named pipe interface. All tests are designed to provide reliable results for up to 100 terabits of test data. Some tests (OQSO, Entropy, and Serial) have approximate test result distributions and the test results will become unreliable for extremely large test data quantities.

Tests Description

• Bits Tested: total number of bits tested
• Elapsed Time: time from the start of the current test run
• Throughput: input data rate in bits per second.
• Bits Tested %: percent of total bits tested. Might be less than 100% due limited CPU resources.
• 1/0 Balance: bias with the expected value p(0) = p(1) = 0.5
• Auto Correlation: orders 1 up to 32, expected value is standard deviation = 0.0 for all orders
• Entropy Test: expected value is H = 1.0, an update of U.
• Maurer’s “Universal Test” (J-S Coron, LNCS, Vol. 1560, pp29-42, 1999)
• Serial Test: expected value is standard deviation = 0.0 (Good, I. J, The serial test for sampling numbers and other tests for randomness, Proc. Camb. Philos. Soc. Vol. 49, 1953)
• OQSO: Overlapping-Quadruples-Sparse-Occupancy test, expected value is standard deviation = 0.0 (G. Marsaglia and A. Zaman, Computers Math. Applic., Vol. 26, No. 9, pp 1-10, 1993)
• Meta KS Test: Kolmogorov Smirnov evaluation of the meta tests. Expected value is 0.5 for both KS+ and KS-

To use the QNGmeter to test ComScire hardware, connect the hardware to the testing computer and press the Start button to begin a new test. To test an external source, first start the QNGmeter and then use the external program to send data through the named pipe. See example code below.

Each test uses blocks of data of varying lengths, depending on the specific test. The 1/0 Balance and Auto Correlation tests use a block size of 65536 bits. The Serial and Entropy tests have a block size of 4194304 bits. The OQSO test uses 10485775 bits per block.

The 1/0 Balance and the Auto Correlation tests have exact binomial solutions and are limited only by the binary resolution of the computational algorithms used. The Serial test is itself a chi-squared test and will asymptotically converge toward the expected result as the block length increases. The OQSO and Entropy tests are empirically determined to be sufficiently normally distributed. All tests are designed to provide reliable results for at least 10¹⁴ bits or more.

Statistical measurements produce probabilistic results with increasing accuracy as the bit count increases. As a result, test results can fall below or above their actual values. For example, it is not uncommon to find an Entropy test (measurement) with an H-value above 1.0. Also note that the Entropy test was not designed for pseudorandom generators in that the test does not measure actual entropy, but statistical entropy. A pseudorandom generator may fail the Entropy test by producing an H-value well above 1.0, thus demonstrating a recognizable generation pattern.

A z-score is calculated for every test for each data-block. The z-scores are converted to probabilities with the assumption they are normally distributed. The z-scores of the 1/0 Balance, Auto Correlation and Serial tests and their associated p-values displayed are cumulative for all blocks. The z-scores of the Entropy and OQSO tests are combined by summing the z-scores of all blocks and dividing by the square root of the number of blocks, respectively.

A second level of testing (meta testing) is applied to the p-values calculated from the z-scores for each block of data. The z-scores are expected to be normally distributed and hence their associated p-values are expected to be uniformly distributed. A chi-square test is applied to the individual p-values from each of the four tests. The chi-square tests are cumulative and their results are displayed as probabilities. If these chi-square p-values converge to 0.0 or 1.0 for any test, the assumption of randomness fails, indicating non-random patterns in the data being tested.

Any test might fail the chi-square test even though it passes the first-level test. A typical example is a pseudo random generator with a “too perfect” 1/0 balance. Note: The chi-square test is an approximation that begins to converge only after about 100 individual probability values have been accumulated.

A third level of testing is applied to all of the individual chi-squared tests. A Kolmogorov-Smirnov (KS) test is first applied to the probabilities of chi-squared results of all orders of auto correlation being tested to reduce the auto correlation results to a single number. A meta KS test is finally calculated using the auto correlation KS(-) result and the probabilities of the chi-squared results of the remaining tests. The meta KS(+) and KS(-) probabilities are displayed. Convergence toward 1.0 or 0.0 indicates failure.

The meter, displayed with a scale from 0 to 50, is an attempt to quantify the quality of the random number generator being tested. The meter value is calculated from the number of bits tested before the generator has been determined to have failed according to a set of failure rules. If b is the number of bits tested before failure, then the meter value is given by log₂(b/4). The 1/4 adjustment is included to estimate the length of “good” bit sequences available from the generator since the data is tested to significant statistical failure.

If any cumulative or meta chi-squared test probability is less than 0.00001 or greater than 0.99999 the generator will be determined as failed and the meter value will “freeze”. It is statistically possible, though unlikely, that a good random sequence can reach the meter freeze condition, therefore, an unfreeze condition is introduced, whereby the meter will declare a false failure and resume if the probability of the failing test returns to a range between 0.01 and 0.99.

The meter value is immediately followed by a plus sign or no sign. A plus indicates that the generator has not failed and its score is at least what is indicated. No sign indicates that the meter is in a frozen state because the generator has failed according to the above rules. If the chi-squared tests have not yet converged, the meter may unfreeze by continuing the test. Otherwise, this score is the final score for the testing generator.

To gain confidence in the meter score for a particular generator, test the generator multiple times. If testing a pseudo random generator, use different seeds for each test.

QNGmeter automatically creates a test report file each time it runs a test. The test report files are created in the Documents/QNGmeter.Data folder. A test report file is a human readable text file and will be named with the serial number of the hardware generator followed by the day’s test date. Or, if the pipe interface is used the test report file will be named “External RNG” followed by the day’s test date. Examples of file names include “QWR3X001_2019.11.15” and “External RNG_2019.11.15.txt”. Note that each successive test for the same generator on the same day will overwrite that day’s test report for that generator. The test report lists general test information, test p-values, and finally meta test p-values.