This article explains what amplitude quantization error is and what can be done to avoid it. Typically quantization error is caused during acquisition of data. It is synonymous to rounding error. In order to explain this, a real time signal has been artificially modified to an extreme to show the difference between a signal sampled with adequate amplitude resolution and the same signal sampled with insufficient amplitude resolution. Also, this example was taken into the frequency domain to demonstrate how quantization error can affect the spectrum of the signal.
Amplitude Quantization Error is typically caused by not having sufficient amplitude resolution to accurately capture the exact amplitude of the signal. With the modern ADC’s using 24 or even 16 bit resolution this is not typically a problem, however measurement systems with fewer bits of amplitude resolution can compromise the data.
The cause of this insufficient resolution is based on the sampling of the level of a signal, or in other words, having too few bits to accurately define the signal level during the sampling process. Typically analog to digital converters have a specified voltage range which they can accept. The Prosig hardware ADC range is ±10 Volts. The ADC’s in these systems are 24-bit which means the total range of 20 Volts (±10 Volts) are 16777216 (224) different levels. This should be more than sufficient number of levels to adequately define the signal level from most transducers. To demonstrate this, I have sampled a signal with adequate amplitude resolution (Figure 1).
However, under rare circumstances if the transducer used has a very low sensitivity (e.g. 1µV/EU) and and the system gain is not adjusted to use the full ADC range, this can lead to the amplitude not having adequate resolution to accurately described the signal. If only ±1 V or less of the ±10 V available of the ADC range is used there are far fewer levels which can be used to define the signal level.
Now the same data artificially modified introducing an extreme amount of rounding error is shown in Figure 2. The levels of approximately -15 EU to +12.5 EU as displayed in Figure 1 has been rounded to quantized levels of -12, -8, -4, 0, 4, 8, and 12.
This, of course, is an exaggeration, but it becomes visually apparent that there is a problem with this data.
What does this mean about the spectrum of a measurement taken with inadequate amplitude resolution? I have processed both these signals using the RMS Auto spectrum analysis tool and overlayed the results (See Figure 3). At first thought one might expect to see this quantization carried over into the spectrum. This is not the case due to how the amplitude of each of the frequency bins are calculated in the FFT calculations.
A significantly higher noise floor is apparent for the signal with the quantization error. This can potentially mask peaks in the spectrum at or below this noise floor (Figure 3).
Taking a closer look, it appears there is a discrepancy in the amplitudes of the two signals (Figure 4). Further investigation into this was to bandpass the time signals (w/o quantization error and w/ quantization error) around the frequencies of the 2 dominant peaks in the spectral data (2 Hz and 58 Hz). The RMS values of each of these 2 bandpass time signals were calculated and there is very close agreement to those values in the spectral calculations (See these values displayed in Figure 4).
Summary and Solutions
There are two quick solutions to the problem of amplitude quantization error. First is to use the appropriate transducer for the range of levels measured with the transducer and the second is to apply gain to the signal to use at least 50% of the available ADC range of the measurement system. In most cases one or both suggestions used together will resolve the situation of inadequate amplitude resolution.
John Mathey graduated with a MS degree from the University of Toledo in 1972. John has over 35 years of experience with instrumentation, measurement, and analysis. Twenty-five of those years were spent at Ford Motor Company solving and providing training for vehicle noise, vibration, and harshness (NVH) issues. He was a technical specialist at Prosig USA, Inc. where he provided technical support to Prosig customers in the U.S.A.