Spatial geographic variability in Earth's core magnetic field, measured at 400 km altitude but corrected for ionospheric and magnetospheric signals, correlates with electron flux levels measured by the RBSP spacecraft. A higher Earth's |B| magnitude results in lower flux over L2-6. Over 20 eV-2 MeV, at L2-4, this negative correlation is as large as -0.21, peaking at the 158 keV electrons, with the strongest effects in the 71 keV-2 MeV electrons. Despite higher L shells being well above the 400 km field measure, statistically significant correlation with the core field was still seen in higher energy 1-2 MeV electrons over L5-6. Adding Earth's geographic |B| variability as a covariate in regression or ARMAX analyses, particularly at lower L shells, results in stronger correlations between electron flux and solar wind, substorm, and ULF wave drivers, with possible nonlinearity in the associations accounted for by taking logs of the variables. At L2, substorms (measured by the SME index), ULF waves, and solar wind velocity show increased correlations with electron flux (30%, 100%, and 175%, respectively) when Earth's |B| is added as a covariate to the ARMAX regression models. Modest increases in correlation of electron flux with these possible drivers were also seen at L3-6. This argues for the addition of Earth's |B| as a covariate in models of electron response to drivers.
Discriminating between volatile organic compounds (VOCs) for applications including disease diagnosis and environmental monitoring, is often complicated by the presence of interfering compounds such as oxygen. Graphene sensors are effective at detecting VOCs; however, they are also known to be highly sensitive to oxygen. Therefore, the combined effects of each of these gases on graphene sensors must be understood. In this work, we use graphene variable capacitor (varactor) sensors to examine the cross-selectivity of oxygen at 3 concentrations and 3 VOCs (ethanol, methanol, and methyl ethyl ketone) at 5 concentrations each. The sensor responses exhibit distinct shapes dependent on the relative concentrations in mixtures of oxygen and VOCs. Because the entire response shape is therefore informative for distinguishing between each gas mixture, a classification algorithm that utilizes entire sequences of data is needed. Accordingly, a long short-term memory (LSTM) network is used to classify the mixtures and VOC concentrations. The model achieves 100% accurate classification of the VOC type, even in the presence of varying levels of oxygen. When the VOC type and VOC concentration are classified, we show that the sensors can provide VOC concentration resolution within approximately 200 ppm. Throughout this work, we also demonstrate that an effective gas mixture classification can be achieved, even while the sensors exhibit varied drift patterns typical of graphene sensors. This is made possible due to the data analysis and machine learning methods employed.
Electron fluxes (20 eV-2 MeV, RBSP-A satellite) show reasonable simple correlation with a variety of parameters (solar wind, IMF, substorms, ultralow frequency (ULF) waves, geomagnetic indices) over L-shells 2-6. Removing correlation-inflating common cycles and trends (using autoregressive and moving average terms in an ARMAX analysis) results in a 10 times reduction in apparent association between drivers and electron flux, although many are still statistically significant (p < 0.05). Corrected influences are highest in the 20 eV-1 keV and 1-2 MeV electrons, more modest in the midrange (2-40 keV). Solar wind velocity and pressure (but not number density), IMF magnitude (with lower influence of B-z), SME (a substorm measure), a ULF wave index, and geomagnetic indices Kp and SymH all show statistically significant associations with electron flux in the corrected individual ARMAX analyses. We postulate that only pressure, ULF waves, and substorms are direct drivers of electron flux and compare their influences in a combined analysis. SME is the strongest influence of these three, mainly in the eV and MeV electrons. ULF is most influential on the MeV electrons. Pressure shows a smaller positive influence and some indication of either magnetopause shadowing or simply compression on the eV electrons. While strictly predictive models may improve forecasting ability by including indirect driver and proxy parameters, and while these models may be made more parsimonious by choosing not to explicitly model time series behavior, our present analyses include time series variables in order to draw valid conclusions about the physical influences of exogenous parameters.
Abstract Regression models (LEEMYR: Low Energy Electron MLT geosYnchronous orbit Regression) predict hourly 4.1–30 keV electron flux at geostationary orbit (GOES‐16) using solar wind, IMF, and geomagnetic index parameters. Multiplicative interaction and polynomial terms describe synergistic and nonlinear effects. We reduce predictors to an optimal set using stepwise regression, resulting in models with validation comparable to a neural network. Models predict 1, 3, 6, 12, and 24 hr into the future. Validation correlations are as high as 0.78 (4.1 and 11 keV, 1 hr prediction) and Heidke Skill scores (HSS) up to 0.66. A 3 hr ahead prediction is more practical, with slightly lower validation correlation (0.75) and HSS (0.61). The addition of location (MLT: magnetic local time) as a covariate, including multiplicative interaction terms, accounts for location‐dependent flux differences and variation of parameter influence, and allows prediction over the full orbit. Adding a substorm index (SME) provides minimal increase in validation correlation (0.81) showing that other parameters are good proxies for an unavailable real time substorm index. Prediction intervals on individual values provide more accurate assessments of model quality than confidence intervals on the mean values. An inverse N‐weighted least squares approach is impractical as it increases false positive warnings. Physical interpretations are not possible as spurious correlations due to common cycles are not removed. However, SME, Bz, Kp, and Dst are the highest correlates of electron flux, with solar wind velocity, density, and pressure, and IMF magnitude being less well correlated.
A supervised convolutional neural network (CNN) was developed to automatically identify electromagnetic ion cyclotron (EMIC) wave events from spectrograms. These events have usually been identified manually, which can be a time-consuming process. Statistical analyses of larger datasets would be facilitated if this process were simplified. The neural network model was trained on spectrogram images from the Halley magnetometer station that had been manually identified as either containing or not containing an EMIC wave event anywhere in the spectrogram. This model was tested on an unseen set of spectrograms, achieving a perfect true positive rate of 1. Size, time, frequency, and pixel color information was extracted from each identified event and exported into a spreadsheet for easier analysis. This method has the capability of reducing time and effort required to identify important spectrogram features by hand. Such an automated method could be applied to other space weather data stored in spectrograms.
Although lagged correlations have suggested influences of solar wind velocity ( V ) and number density ( N ), Bz, ultralow frequency (ULF) wave power, and substorms (as measured by the auroral electrojet (AE) index) on MeV electron flux at geosynchronous orbit over an impressive number of hours and days, a satellite's diurnal cycle can inflate correlations, associations between drivers may produce spurious effects, and correlations between all previous time steps may create an appearance of additive influence over many hours. Autoregressive‐moving average transfer function (ARMAX) multiple regressions incorporating previous hours simultaneously can eliminate cycles and assess the impact of parameters, at each hour, while others are controlled. ARMAX influences are an order of magnitude lower than correlations uncorrected for time behavior. Most influence occurs within a few hours, not the many hours suggested by correlation. A log transformation accounts for nonlinearities. Over all hours, solar wind velocity ( V ) and number density ( N ) show an initial negative impact, with longer term positive influences over the 9 ( V ) or 27 ( N ) hr. Bz is initially a positive influence, with a longer term (6 hr) negative effect. ULF waves impact flux in the first (positive) and second (negative) hour before the flux measurement, with further negative influences in the 12–24 hr before. AE (representing electron injection by substorms) shows only a short term (1 hr) positive influence. However, when only recovery and after‐recovery storm periods are considered (using stepwise regression), there are positive influences of ULF waves, AE, and V , with negative influences of N and Bz.
Abstract We screen several algorithms for their ability to produce good predictive models of hourly 40–150 keV electron flux at geostationary orbit (data from GOES‐13) using solar wind, Interplanetary Magnetic Field, and geomagnetic index parameters that would be available for real time forecasting. Value‐predicting models developed using ARMAX (autoregressive moving average transfer function), RNN (recurrent neural network), or stepwise‐reduced regression produced roughly similar results. Including magnetic local time as a categorical variable to describe both the differing levels of flux and the differing influence of parameters improved the models (r as high as 0.814; Heidke Skill Score (HSS) as high as 0.663), however value‐predicting models did a poor job at predicting highs and lows. Diagnostic tests are introduced (cubic fit to observation‐prediction relationship and Lag1 correlation) that better assess predictions of extremes than single metrics such as root mean square error, mean absolute error, or median symmetric accuracy. Classifier models (RNN and logistic regression) were equally able to predict flux rise above the 75th percentile (HSS as high as 0.667). Logistic regression models were improved by the addition of multiplicative interaction and quadratic terms. Only predictors from 1 or 3 hr before were necessary and a detailed description of flux time series behavior was not needed. Stepwise selection of these variables trimmed non‐contributing parameters for a more parsimonious and portable logistic regression model that predicted as well as neural network‐derived models. We provide a logistic regression model (LL3: LogisticLag3) based on inputs measured 3 hr previous, along with optimal probability thresholds, for future predictions.
Earth and Space Science Open Archive This preprint has been submitted to and is under consideration at Journal of Geophysical Research - Space Physics. ESSOAr is a venue for early communication or feedback before peer review. Data may be preliminary.Learn more about preprints preprintOpen AccessYou are viewing the latest version by default [v1]Using ARMAX models to determine the drivers of 40-150 keV GOES electron fluxesAuthorsLaura E.SimmsiDNatalia YuGanushkinaMaxvan de KampMichael W.LiemohniDStepanDubyaginSee all authors Laura E. SimmsiDCorresponding Author• Submitting AuthorDepartment of Physics, Augsburg UniversityiDhttps://orcid.org/0000-0002-2934-8823view email addressThe email was not providedcopy email addressNatalia Yu GanushkinaFinnish Meteorological Instituteview email addressThe email was not providedcopy email addressMax van de KampFinnish Meteorological Instituteview email addressThe email was not providedcopy email addressMichael W. LiemohniDUniversity of Michigan-Ann ArboriDhttps://orcid.org/0000-0002-7039-2631view email addressThe email was not providedcopy email addressStepan DubyaginFinnish Meteorological Instituteview email addressThe email was not providedcopy email address
We investigate the drivers of 40-150 keV hourly electron flux at geostationary orbit (GOES 13) using autoregressive moving average transfer functions (ARMAX) multiple regression models which remove the confounding effect of diurnal cyclicity and allow assessment of each parameter independently. By taking logs of the variables, we create nonlinear models. While many factors show high correlation with flux in single variable analysis (substorms, ULF waves, solar wind velocity (V), pressure (P), number density (N) and electric field (E-y), IMF Bz, Kp, and SymH), ARMAX models show substorms are the dominant influence at 40-75 keV and over 20-12 MLT, with little difference seen between disturbed and quiet periods. The Ey influence is positive post-midnight, negative post-noon. Pressure shows a negative influence, strongest at 150 keV. ULF waves are a more modest influence than suggested by single variable correlation. Kp and SymH show little effect when other variables are included. Using path analysis, we calculate the summed direct and indirect influences through the driving of intermediate parameters. Pressure shows a summed direct and indirect influence nearly half that of the direct substorm effect. N, V, and B-z, as indirect drivers, are equally influential. While simple correlation or neural networks can be used for flux prediction, neither can effectively identify drivers. Instead, consideration of physical influences, removing cycles that artificially inflate correlations, and controlling the effects of other parameters gives a clearer picture of which are most influential in this system.
Nearly all studies of impulsive geomagnetic disturbances (GMDs, also known as magnetic perturbation events MPEs) that can produce dangerous geomagnetically induced currents (GICs) have used data from the northern hemisphere. In this study, we investigated GMD occurrences during the first 6 months of 2016 at four magnetically conjugate high latitude station pairs using data from the Greenland West Coast magnetometer chain and from Antarctic stations in the conjugate AAL‐PIP magnetometer chain. Events for statistical analysis and four case studies were selected from Greenland/AAL‐PIP data by detecting the presence of >6 nT/s derivatives of any component of the magnetic field at any of the station pairs. For case studies, these chains were supplemented by data from the BAS‐LPM chain in Antarctica as well as Pangnirtung and South Pole in order to extend longitudinal coverage to the west. Amplitude comparisons between hemispheres showed (a) a seasonal dependence (larger in the winter hemisphere), and (b) a dependence on the sign of the By component of the interplanetary magnetic field (IMF): GMDs were larger in the north (south) when IMF By was >0 (<0). A majority of events occurred nearly simultaneously (to within ±3 min) independent of the sign of By as long as | By | ≤ 2 | Bz |. As has been found in earlier studies, IMF Bz was <0 prior to most events. When IMF data from Geotail, Themis B, and/or Themis C in the near‐Earth solar wind were used to supplement the time‐shifted OMNI IMF data, the consistency of these IMF orientations was improved.
Simultaneously cycling space weather parameters may show high correlations even if there is no immediate relationship between them. We successfully remove diurnal cycles using spectral subtraction, and remove both diurnal and longer cycles (e.g., the 27 days solar cycle) with a difference transformation. Other methods of diurnal cycle removal (daily averaging, moving averages [MAs], and simpler spectral subtraction using regression) are less successful at removing cycles. We apply spectral subtraction (a finite impulse response equiripple bandstop filter) to hourly electron flux (Los Alamos National Laboratory satellite data) and a ground‐based ULF index to remove a 24 hr noise signal. This results in smoother time series appropriate for short‐term (approximately < 1 week) correlation and observational studies. However, spectral subtraction may not remove longer cycles such as the 27 days and 11 yr solar cycles. A differencing transformation ( y t – y t−24 ) removes not only the 24 hr noise signal but also the 27 days solar cycle, autocorrelation, and longer trends. This results in a low correlation between electron flux and the ULF index over long periods of time (maximum of 0.1). Correlations of electron flux and the ULF index with solar wind velocity (differenced at y t – y t−1 ) are also lower than previously reported (≤0.1). An autoregressive, MA transfer function model (ARIMAX) shows that there are significant cumulative effects of solar wind velocity on ULF activity over long periods, but correlations of velocity and ULF waves with flux are only seen over shorter time spans of more homogeneous geomagnetic activity levels.
We investigate the timing and relative influence of VLF in the chorus frequency range observed by the DEMETER spacecraft and ULF wave activity from ground stations on daily changes in electron flux (0.23 to over 2.9 MeV) observed by the HEO‐3 spacecraft. At each L‐shell, we use multiple regression to investigate the effects of each wave type and each daily lag independent of the others. We find that reduction and enhancement of electrons occur at different timescales. Chorus power spectral density and ULF wave power are associated with immediate electron decreases on the same day but with flux enhancement 1–2 days later. ULF is nearly always more influential than chorus on both increases and decreases of flux, although chorus is often a significant factor. There was virtually no difference in correlations of ULF Pc3, Pc4, or Pc5 with electron flux. A synergistic interaction between chorus and ULF waves means that enhancement is most effective when both waves are present, pointing to a two‐step process where local acceleration by chorus waves first energizes electrons which are then brought to even higher energies by inward radial diffusion due to ULF waves. However, decreases in flux due to these waves act additively. Chorus and ULF waves combined are most effective at describing changes in electron flux at >1.5 MeV. At lower L (2–3), correlations between ULF and VLF (likely hiss) with electron flux were low. The most successful models, over L = 4–6, explained up to 47.1% of the variation in the data.
To find the best method of predicting when daily relativistic electron flux (>2 MeV) will rise at geosynchronous orbit, we compare model predictive success rates (true positive rate or TPR) for multiple regression, ARMAX, logistic regression, a feed-forward multilayer perceptron (MLP), and a recurrent neural network (RNN) model. We use only those days on which flux could rise, removing days when flux is already high from the data set. We explore three input variable sets: (1) ground-based data (Kp, Dst, and sunspot number), (2) a full set of easily available solar wind and interplanetary magnetic field parameters (|B|, Bz, V, N, P, Ey, Kp, Dst, and sunspot number, and (3) this full set with the addition of previous day's flux. Despite high validation correlations in the multiple regression and ARMAX predictions, these regression models had low predictive ability (TPR < 45%) and are not recommended for use. The three classifier model types (logistic regression, MLP, and RNN) performed better (TPR: 50.8-74.6%). These rates were increased further if the cost of missing an event was set at 4 times that of predicting an event that did not happen (TPR: 73.1-89.6%). The area under the receiver operating characteristic curves did not, for the most part, differ between the classifier models (logistic, MLP, and RNN), indicating that any of the three could be used to discriminate between events and nonevents, but validation suggests a full RNN model performs best. The addition of previous day's flux as a predictor provided only a slight advantage. Key Points Multiple regression and ARMAX models performed much worse than classifier models at predicting relativistic electron flux Of the three classifier models (logistic regression, MLP, and RNN) RNN performed best Persistence (previous day's flux) is not a necessary predictor
Ground observations of VLF (very low frequency) waves have often been used to infer VLF activity in the magnetosphere; however, they are not an unbiased measure of activity at satellite altitudes due to transionospheric absorption and subionospheric attenuation. We propose several empirical models that control for these effects. VLF power spectral density (PSD) from the VLF/ELF Logger Experiment (VELOX, L=4.6, Halley, Antarctica) is used to predict DEMETER low Earth orbit VLF PSD. Validation correlations of these models are as high as 0.764; thus, ground VLF receivers spaced around the Earth could provide coverage of outer radiation belt lower band chorus over the latitudinal limits of this model (±45–75°). Correlations of four frequency bands (centered at 0.5, 1.0, 2.0, and 4.25 kHz) are compared. The simple linear correlation between ground and satellite VLF PSD in the 1.0‐kHz channel was 0.606 (at dawn). A cubic model resulted in higher correlation (0.638). VLF penetration to the ground is reduced by ionospheric absorption during solar illumination and by disruption of ducting field lines during disturbed conditions. Subionospheric attenuation also reduces VLF observations from distant field lines. Addition of these covariates improved predictions. Both solar illumination and disturbed conditions reduced ground observation of VLF PSD, with higher power waves penetrating to the ground proportionately less than lower power waves. The effect of illumination in reducing wave penetration was more pronounced at higher frequency (4.25 kHz), with the effect at a midrange frequency (2.0 kHz) falling between these two extremes.
We model lower band chorus observations from the DEMETER satellite using daily and hourly autoregressive‐moving average transfer function (ARMAX) equations. ARMAX models can account for serial autocorrelation between observations that are measured close together in time and can be used to predict a response variable based on its past behavior without the need for recent data. Unstable distributions of radiation belt source electrons (tens of keV) and the substorm activity (SMEd from the SuperMAG array) that is thought to inject these electrons were both statistically significant explanatory variables in a daily ARMAX model describing chorus. Predictions from this model correlated well with observations in a hold‐out test data set (validation correlation of 0.675). Source electron flux was most influential when observations came from the same day or the day before the chorus measurement, with effects decaying rapidly over time. Substorms were more influential when they occurred on previous days, presumably due to their injecting source electrons from the plasma sheet. A daily ARMAX model with interplanetary magnetic field (IMF)|B|, IMF Bz, and solar wind pressure as inputs instead of those given above was somewhat less predictive of chorus (r=0.611). An hourly ARMAX model with only solar wind and IMF inputs was even less successful, with a validation correlation of 0.502.
Many factors influence relativistic outer radiation belt electron fluxes, such as waves in the ultralow frequency (ULF) Pc5, very low frequency (VLF), and electromagnetic ion cyclotron (EMIC) frequency bands, seed electron flux, Dst disturbance levels, substorm occurrence, and solar wind inputs. In this work we compared relativistic electron flux poststorm versus prestorm using three methods of analysis: (1) multiple regression to predict flux values following storms, (2) multiple regression to predict the size and direction of the change in electron flux, and (3) multiple logistic regression to predict only the probability of the flux rising or falling. We determined which is the most predictive model and which factors are most influential. We found that a linear regression predicting the difference in prestorm and poststorm flux (Model 2) results in the highest validation correlations. The logistic regression used in Model 3 had slightly weaker predictive abilities than the other two models but had the most value in providing a prediction of the probability of the electron flux increasing after a storm. Of the variables used (ULF Pc5 and VLF, seed electrons, substorm activity, and EMIC waves), the most influential in the final model were ULF Pc5 waves and the seed electrons. IMF Bz, Dst, and solar wind number density, velocity, and pressure did not improve any of the models, and were deemed unnecessary for effective predictions.
18 Ground VLF observations have often been used to infer VLF activity in the magnetosphere, however, 19 they are not an unbiased measure of activity at satellite altitudes due to transionospheric absorption 20 and subionospheric attenuation. We propose several empirical models that control for these effects. 21 VLF power spectral density (PSD) from the VLF/ELF Logger Experiment (VELOX, L=4.6, Halley, Antarctica) 22 is used to predict DEMETER low Earth orbit VLF PSD. Validation correlations of these models are as high 23 as 0.764, thus ground VLF receivers spaced around the Earth could provide complete coverage of outer 24 radiation belt lower band chorus over the latitudinal limits of this model (±45-75°). Correlations of four 25 frequency bands (centered at 0 .5 kHz, 1.0 kHz, 2.0 kHz, and 4.25 kHz) are compared. The simple linear 26 correlation between ground and satellite VLF PSD in the 1.0 kHz channel was 0.606 (at dawn). A cubic 27 model resulted in higher correlation (0.638). VLF penetration to the ground is reduced by ionospheric 28 absorption during solar illumination and by disruption of ducting field lines during disturbed conditions. 29 Subionospheric attenuation also reduces VLF observations from distant field lines. Addition of these 30 covariates improved predictions. Both solar illumination and disturbed conditions reduced ground 31 observation of VLF PSD, with higher power waves penetrating to the ground proportionately less than 32 lower power waves. The effect of illumination in reducing wave penetration was more pronounced at 33 higher frequency (4.25 kHz), with the effect at a mid-range frequency (2.0 kHz) falling between these 34 two extremes. 35
Relativistic electron flux at geosynchronous orbit depends on enhancement and loss processes driven by ultralow frequency (ULF) Pc5, chorus, and electromagnetic ion cyclotron (EMIC) waves, seed electron flux, magnetosphere compression, the "Dst effect", and substorms, while solar wind inputs such as velocity, number density, and interplanetary magnetic field Bz drive these factors and thus correlate with flux. Distributed lag regression models show the time delay of highest influence of these factors on log(10) high-energy electron flux (0.7-7.8 MeV, Los Alamos National Laboratory satellites). Multiple regression with an autoregressive term (flux persistence) allows direct comparison of the magnitude of each effect while controlling other correlated parameters. Flux enhancements due to ULF Pc5 and chorus waves are of equal importance. The direct effect of substorms on high-energy electron flux is strong, possibly due to injection of high-energy electrons by the substorms themselves. Loss due to electromagnetic ion cyclotron waves is less influential. Southward Bz shows only moderate influence when correlated processes are accounted for. Adding covariate compression effects (pressure and interplanetary magnetic field magnitude) allows wave-driven enhancements to be more clearly seen. Seed electrons (270 keV) are most influential at lower relativistic energies, showing that such a population must be available for acceleration. However, they are not accelerated directly to the highest energies. Source electrons (31.7 keV) show no direct influence when other factors are controlled. Their action appears to be indirect via the chorus waves they generate. Determination of specific effects of each parameter when studied in combination will be more helpful in furthering modeling work than studying them individually.
Using data covering the years 2005–2009, we study the linear and nonlinear responses of log10 relativistic electron flux measured at geosynchronous orbit to ultralow frequency (ULF) Pc5, very low frequency (VLF) lower band chorus, and electromagnetic ion cyclotron (EMIC) waves. We use regression models incorporating a quadratic term and a synergistic interaction term. Relativistic electron fluxes respond to ULF Pc5 and VLF chorus waves both linearly and nonlinearly. ULF Pc5 waves contribute both to electron enhancement (at midrange wave activity) and loss (at high levels of wave activity). Nonlinear effects of VLF chorus are positive (i.e., cause acceleration), adding to the positive linear effects. Synergistic interaction effects between high levels of VLF chorus and midrange values of ULF Pc5 waves result in more electron acceleration than would be predicted by a simpler additive model. Similarly, the negative effect of EMIC waves (losses) is more influential than would be predicted by a linear model when combined with either VLF chorus or ULF Pc5 waves. During disturbed conditions (high Kp), geostationary electron flux responds more strongly to the same levels of ULF Pc5 and VLF chorus waves. This flux also responds more to ULF Pc5 and chorus waves during southward Bz conditions. Unstandardized regression coefficients for models incorporating nonlinear and synergistic effects of waves are presented for use in future modeling.