Don’t Let Spikes Spoil Your Data

In many real-world applications it is impossible to avoid “spikes” or “dropouts” in data that we record. Many people assume that these only cause problems with their data if they become obvious. This is not always the case. Consider the following two time histories.

Figure 1: Signal containing "spikes"

Figure 2: Signal with "spikes" removed

To the naked eye they look identical, but closer inspection reveals a few small “spikes” in the first signal. These can be seen when overlaid and zoomed, but they are not at all obvious. Figure 3 shows a “zoomed in” view of the two signals overlaid. The green curve overlays the blue one precisely except for the “spike” at around t=7.624seconds.

Figure 3: A detail of both signals overlaid

Although there were only four such “spikes” in a signal of over 44000 points they affect (infect?) the frequency spectrum. Figure 4 below shows the averaged frequency spectra of the original and “despiked” signals. The blue trace is the spectrum of the original signal and it can be seen that it flattens out to about 23 and that we get no information other than noise beyond about 4kHz. That is we have a noise floor at around 23dB.

Figure 4: Frequency spectra of original and "despiked" data

The red trace is the averaged frequency spectrum of the signal after “despiking” and one can see that we now have information across the entire frequency range. Significantly, there is potentially important information at around 11 to 12kHz. In fact, it was these very small responses that we were looking for. Later, as the fault develops the energy in that frequency region grows. The spikes have lost about 20dB of dynamic range which is almost like losing 3 bits of resolution from the analogue to digital converter. It also delays the time before the fault was detected.In the above example, the DATS REMSPIKE function was used to remove the spikes from the original signal. This is done using a sophisticated algorithm in one analysis step, removing the need to tediously examine and “despike” the data manually.

The following two tabs change content below.

Dr Colin Mercer

Chief Signal Processing Analyst at Prosig
Dr Colin Mercer was formerly at the Institute of Sound and Vibration Research (ISVR), University of Southampton where he founded the Data Analysis Centre. He then went on to found Prosig in 1977. Colin retired as Chief Signal Processing Analyst at Prosig in December 2016. He is a Chartered Engineer and a Fellow of the British Computer Society.

Latest posts by Dr Colin Mercer (see all)

Leave a Reply

  1. We welcome any feedback, questions or comments
CLOSE
CLOSE
Optimization WordPress Plugins & Solutions by W3 EDGE