Automated High-Frequency Geomagnetic Disturbance Classifier: A Machine Learning Approach to Identifying Noise While Retaining High-Frequency Components of the Geomagnetic Field

We present an automated method to identify high-frequency geomagnetic disturbances in ground magnetometer data and classify the events by the source of the perturbations. We developed an algorithm to search for and identify changes in the surface magnetic field, dB/dt, with user-specified amplitude and timescale. We used this algorithm to identify transient-large-amplitude (TLA) dB/dt events that have timescale less than 60 s and amplitude > 6 nT/s. Because these magnetic variations have similar amplitude and time characteristics to instrumental or man-made noise, the algorithm identified a large number of noise-type signatures as well as geophysical signatures. We manually classified these events by their sources (noise-type or geophysical) and statistically characterized each type of event; the insights gained were used to more specifically define a TLA geophysical event and greatly reduce the number of noise-type dB/dt identified. Next, we implemented a support vector machine classification algorithm to classify the remaining events in order to further reduce the number of noise-type dB/dt in the final data set. We examine the performance of our complete dB/dt search algorithm in widely used magnetometer databases and the effect of a common data processing technique on the results. The automated algorithm is a new technique to identify geomagnetic disturbances and instrumental or man-made noise, enabling systematic identification and analysis of space weather related dB/dt events and automated detection of magnetometer noise intervals in magnetic field databases.High-frequency (second-timescale) components of the surface geomagnetic field are not often included in studies on geomagnetically induced currents (GICs) because they do not pose a direct threat to technological infrastructure. However, high-frequency intervals occur prior to and within some larger space weather events that can lead to GICs. Because these perturbations are very similar to signals that arise due to noise-interference, we have developed an automated procedure to identify such high-frequency intervals and predict the source of the signal as geophysical or noise-type. It was found that common data processing techniques can reduce or remove high-frequency geophysical disturbances, but do not remove all noise-type intervals. Thus, the automated process provides an event list of 1-hr event windows that contain high-frequency disturbances and the classification of the signals within. This list can be used to identify hour windows of data that are undesirable for space weather research as well as events that contain high-frequency geophysical disturbances that may provide insight to the small-scale features of space weather events.