QNGmeter Continuous Real-Time Statistical Tester
The QNGmeter is a continuous real-time statistical tester that uses five powerful and fundamentally different tests on the input data. Unlike other statistical test suites, it is designed to measure the quality of randomness of a continuous sequence of bits up to hundreds of terabits in length (real example shown below). The QNGmeter automatically performs metatests of subsequences, which would have to be done manually using other popular test suites.
The five tests are:
- 1/0 Balance – nominal expected value is p(1) = p(0) = 0.5.
- Auto Correlation – orders 1 through 32, nominal expected value is 0.5 for all orders.
- Entropy Test – nominal expected value is H = 1.0, an update of U. Maurer’s “Universal Test” [Cor99].
- Serial Test – (Good, I. J, The serial test for sampling numbers and other tests for randomness, Proc. Camb. Philos. Soc. Vol. 49, 1953).
- OQSO – Overlapping-Quadruples-Sparse-Occupancy test, nominal expected value for the mean = 141909.47 and standard deviation (by simulation) = 294.656 (G. Marsaglia and A. Zaman, Computers Math. Applic., Vol. 26, No. 9, pp 1-10, 1993).
The z-scores, p-values, and chi-square (metatest) p-values are presented for each test. In addition, current test run time information, such as Bits Tested, Elapsed Time, Throughput, and Bits Tested %, is displayed by the tester. Bits Tested is the total number of bits tested. Elapsed Time is the time from the start of the current test run. Throughput is the input data rate in bits per second. Bits Tested % is the percent of the total bits tested. This value might be less than 100% due to limited CPU resources.
Each test uses blocks of data of varying lengths, depending on the specific test. The 1/0 Balance and Auto Correlation tests use a block size of 65,536 bits. The Serial test has a block size of 262,144 bits. The Entropy test has 4,194,304 bits in a block. The OQSO test uses 10,485,775 bits per block.
A z-score is calculated for every test for each data-block. The z-scores are converted to probabilities with the assumption they are normally distributed. The z-scores of the 1/0 Balance, Auto Correlation and Serial tests and their associated p-values displayed are cumulative for all blocks. The z-scores of the Entropy and OQSO tests are combined by summing the z-scores of all blocks and dividing by the square root of the number of blocks, respectively.
A second level of testing is applied to the p-values calculated from the z-scores for each block of data. The z-scores are expected to be normally distributed and their associated p-values are expected to be uniformly distributed. A chi-square test is applied to the individual p-values from each of the five tests. The chi-square tests are cumulative and their results are displayed as probabilities. If these chi-square p-values converge to 0.0 or 1.0 for any test, the assumption of randomness fails, indicating non-random patterns in the data being tested.
A third level of testing is applied to all of the individual chi-squared tests. A Kolmogorov-Smirnov (KS) test is first applied to the probabilities of chi-squared results of all orders of auto correlation being tested to reduce the auto correlation results to a single probability. A meta-meta KS test is finally calculated using the auto correlation KS result and the probabilities of the chi-squared metatest results of all the other tests. The meta-meta KS+ and KS- probabilities are displayed. Convergence toward 1.0 or 0.0 indicates failure.
The tester is available with the latest QNG software for our hardware generators. A Windows standalone copy is available for download below. The source code of the tester (QNGmeter Console) is included in the Linux version of the interface software.

QNGmeter 3.6
The QNGmeter is a continuous statistical tester that uses five powerful and fundamentally different tests on the input data. This version tests QNG hardware as well as external sources through a named pipe interface. All tests are designed to provide reliable results for continuous testing up to 500 or more terabits. No other test suites have this capability.
Note: This is the same tester that comes with the latest QNG software. The full source code is available within the Linux version of our interface program.
Free Copyrighted Software
By downloading or distributing any free software provided by Core Invention Inc. you agree to the following terms:
You may freely use, copy and distribute the Software in its original form only, including our copyright notice. You may not reverse engineer, decompile, or disassemble the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
These terms shall be included in all copies or substantial portions of the Software.
- QNGmeter 3.6* (QNGmeter.exe executable program)
- QNGmeter 3.6 Documentation – See below
* QNGmeter Version 3.6 adds support for CryptoStrong™ QNG models.
Previous versions:
* QNGmeter Version 3.5.3 has an update to the external interface.
* QNGmeter Version 3.5.2 adds support for QNG Model PQ128MU.
* QNGmeter Version 3.5.1 increases the range of the OQSO meta test to 100 Tb or more.
QNGmeter 3.6 Documentation
v2019.11.15 © 2019 The Quantum World Corporation.
Overview
The QNGmeter is a continuous statistical tester that uses five powerful and fundamentally different tests on the input data. This version tests ComScire QNG hardware as well as external sources through a named pipe interface. All tests are designed to provide reliable results for up to 100 terabits of test data. Some tests (OQSO, Entropy, and Serial) have approximate test result distributions and the test results will become unreliable for extremely large test data quantities.
Tests Description
- Bits Tested: total number of bits tested
- Elapsed Time: time from the start of the current test run
- Throughput: input data rate in bits per second.
- Bits Tested %: percent of total bits tested. Might be less than 100% due limited CPU resources.
- 1/0 Balance: bias with the expected value p(0) = p(1) = 0.5
- Auto Correlation: orders 1 up to 32, expected value is standard deviation = 0.0 for all orders
- Entropy Test: expected value is H = 1.0, an update of U.
- Maurer’s “Universal Test” (J-S Coron, LNCS, Vol. 1560, pp29-42, 1999)
- Serial Test: expected value is standard deviation = 0.0 (Good, I. J, The serial test for sampling numbers and other tests for randomness, Proc. Camb. Philos. Soc. Vol. 49, 1953)
- OQSO: Overlapping-Quadruples-Sparse-Occupancy test, expected value is standard deviation = 0.0 (G. Marsaglia and A. Zaman, Computers Math. Applic., Vol. 26, No. 9, pp 1-10, 1993)
- Meta KS Test: Kolmogorov Smirnov evaluation of the meta tests. Expected value is 0.5 for both KS+ and KS-
To use the QNGmeter to test ComScire hardware, connect the hardware to the testing computer and press the Start button to begin a new test. To test an external source, first start the QNGmeter and then use the external program to send data through the named pipe. See example code below.
Each test uses blocks of data of varying lengths, depending on the specific test. The 1/0 Balance and Auto Correlation tests use a block size of 65536 bits. The Serial and Entropy tests have a block size of 4194304 bits. The OQSO test uses 10485775 bits per block.
The 1/0 Balance and the Auto Correlation tests have exact binomial solutions and are limited only by the binary resolution of the computational algorithms used. The Serial test is itself a chi-squared test and will asymptotically converge toward the expected result as the block length increases. The OQSO and Entropy tests are empirically determined to be sufficiently normally distributed. All tests are designed to provide reliable results for at least 1014 bits or more.
Statistical measurements produce probabilistic results with increasing accuracy as the bit count increases. As a result, test results can fall below or above their actual values. For example, it is not uncommon to find an Entropy test (measurement) with an H-value above 1.0. Also note that the Entropy test was not designed for pseudorandom generators in that the test does not measure actual entropy, but statistical entropy. A pseudorandom generator may fail the Entropy test by producing an H-value well above 1.0, thus demonstrating a recognizable generation pattern.
A z-score is calculated for every test for each data-block. The z-scores are converted to probabilities with the assumption they are normally distributed. The z-scores of the 1/0 Balance, Auto Correlation and Serial tests and their associated p-values displayed are cumulative for all blocks. The z-scores of the Entropy and OQSO tests are combined by summing the z-scores of all blocks and dividing by the square root of the number of blocks, respectively.
A second level of testing (meta testing) is applied to the p-values calculated from the z-scores for each block of data. The z-scores are expected to be normally distributed and hence their associated p-values are expected to be uniformly distributed. A chi-square test is applied to the individual p-values from each of the four tests. The chi-square tests are cumulative and their results are displayed as probabilities. If these chi-square p-values converge to 0.0 or 1.0 for any test, the assumption of randomness fails, indicating non-random patterns in the data being tested.
Any test might fail the chi-square test even though it passes the first-level test. A typical example is a pseudo random generator with a “too perfect” 1/0 balance. Note: The chi-square test is an approximation that begins to converge only after about 100 individual probability values have been accumulated.
A third level of testing is applied to all of the individual chi-squared tests. A Kolmogorov-Smirnov (KS) test is first applied to the probabilities of chi-squared results of all orders of auto correlation being tested to reduce the auto correlation results to a single number. A meta KS test is finally calculated using the auto correlation KS(-) result and the probabilities of the chi-squared results of the remaining tests. The meta KS(+) and KS(-) probabilities are displayed. Convergence toward 1.0 or 0.0 indicates failure.
The meter, displayed with a scale from 0 to 50, is an attempt to quantify the quality of the random number generator being tested. The meter value is calculated from the number of bits tested before the generator has been determined to have failed according to a set of failure rules. If b is the number of bits tested before failure, then the meter value is given by log2(b/4). The 1/4 adjustment is included to estimate the length of “good” bit sequences available from the generator since the data is tested to significant statistical failure.
If any cumulative or meta chi-squared test probability is less than 0.00001 or greater than 0.99999 the generator will be determined as failed and the meter value will “freeze”. It is statistically possible, though unlikely, that a good random sequence can reach the meter freeze condition, therefore, an unfreeze condition is introduced, whereby the meter will declare a false failure and resume if the probability of the failing test returns to a range between 0.01 and 0.99.
The meter value is immediately followed by a plus sign or no sign. A plus indicates that the generator has not failed and its score is at least what is indicated. No sign indicates that the meter is in a frozen state because the generator has failed according to the above rules. If the chi-squared tests have not yet converged, the meter may unfreeze by continuing the test. Otherwise, this score is the final score for the testing generator.
To gain confidence in the meter score for a particular generator, test the generator multiple times. If testing a pseudo random generator, use different seeds for each test.
QNGmeter automatically creates a test report file each time it runs a test. The test report files are created in the Documents/QNGmeter.Data folder. A test report file is a human readable text file and will be named with the serial number of the hardware generator followed by the day’s test date. Or, if the pipe interface is used the test report file will be named “External RNG” followed by the day’s test date. Examples of file names include “QWR3X001_2019.11.15” and “External RNG_2019.11.15.txt”. Note that each successive test for the same generator on the same day will overwrite that day’s test report for that generator. The test report lists general test information, test p-values, and finally meta test p-values.
External Interface
The following example uses the QNGmeter external pipe interface to test the internal Visual C++ pseudo-random generator. The name of the QNGmeter external interface pipe is “QNGmeter” with a full path of “\\.\pipe\QNGmeter”. Any environment that can write binary data to a named pipe can use the interface for testing. Note that the QNGmeter requires a minimum of 8192 bytes (65536 bits) of data to perform the minimum bias and autocorrelation tests. For accurate testing it is desirable to preserve the correct bit and byte order of the test data. QNGmeter tests bytes in FIFO order and each bit within a byte in LSB to MSB order.
C++ Example
#include
int main() {
UINT8 randByte;
DWORD bytesWritten;
// seed the PRNG with tick count clock
srand(GetTickCount());
// open pipe \\.\pipe\QNGmeter in binary write mode
HANDLE pipeHandle = CreateFile(
L"\\\\.\\pipe\\QNGmeter",
GENERIC_WRITE, 0, NULL, OPEN_EXISTING, 0, NULL
);
while (true) {
// just writing the least significant byte of the pseudo-random word
randByte = (UINT8)rand();
BOOL writeSuccess = WriteFile(pipeHandle, &randByte, 1, &bytesWritten, NULL);
// if anything seems wrong (possible broken pipe, etc), stop the test
if (writeSuccess == FALSE)
break;
}
// close pipe
CloseHandle(pipeHandle);
return 0;
}
